diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6569790..a68c4f9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,6 +4,7 @@ on: pull_request: branches: - main + - staging # `labeled` lets reviewers opt a PR into integration / GPU coverage by # adding `run-integration` or `run-gpu` to the PR. Per-job `if:` checks # the label set, so unrelated jobs still skip. @@ -62,6 +63,56 @@ jobs: - name: Pytest (test_paillier_encryption) run: uv run pytest tests/x_vec/test_paillier_encryption.py + - name: Build and test native BFV CPU kernels + run: | + sudo apt-get update + sudo apt-get install -y libgmp-dev valgrind clang + make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" + uv run python -c 'from xtrace_sdk.x_vec.crypto.bfv_cpu_ext import _bfv_rns, _bfv_private; assert _bfv_rns.ABI_VERSION == 4; assert _bfv_private.ABI_VERSION == 1' + uv run pytest tests/x_vec/test_bfv_encryption.py tests/x_vec/test_bfv_evaluator.py tests/x_vec/test_bfv_rns.py tests/x_vec/test_bfv_native.py tests/x_vec/test_bfv_residue.py tests/x_vec/test_bfv_client.py tests/x_vec/test_bfv_security.py tests/x_vec/test_bfv_verified_client.py tests/x_vec/test_bfv_private.py tests/x_vec/test_bfv_guarded_client.py tests/x_vec/test_bfv_assurance.py + + - name: BFV Nitro attestation and protocol regressions (offline) + run: uv run --extra bfv-nitro pytest tests/x_vec/test_bfv_nitro.py tests/x_vec/test_bfv_attested_client.py tests/x_vec/test_bfv_nitro_transport.py tests/x_vec/test_bfv_nitro_aws.py + + - name: SEAL comparison (synthetic inputs only) + run: | + uv pip install --python .venv/bin/python -r experiments/bfv/requirements.txt + .venv/bin/python -m pytest experiments/bfv/test_packed_hamming.py tests/x_vec/test_bfv_client.py::test_optional_seal_reference + + - name: Build documentation + run: uv run --extra bfv-nitro sphinx-build -W -b html docs /tmp/xtrace-docs + + - name: BFV native arithmetic sanitizers + run: | + g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer -I src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext tests/x_vec/native/test_bfv_rns.cpp -lgmpxx -lgmp -o /tmp/test_bfv_rns + /tmp/test_bfv_rns + g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer -I src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext tests/x_vec/native/test_bfv_private.cpp -lgmpxx -lgmp -o /tmp/test_bfv_private + /tmp/test_bfv_private + + - name: BFV private kernel secret-taint regression + run: | + bfv_gcc_install=$(dirname "$(g++ -print-libgcc-file-name)") + for bfv_cxx in g++ clang++; do + bfv_toolchain=() + if [ "$bfv_cxx" = clang++ ]; then + bfv_toolchain=("--gcc-install-dir=$bfv_gcc_install") + fi + "$bfv_cxx" "${bfv_toolchain[@]}" -std=c++17 -O3 -g -fPIC -fstack-protector-strong -D_FORTIFY_SOURCE=3 -DXTRACE_BFV_CTGRIND -I src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext tests/x_vec/native/test_bfv_private.cpp -lgmpxx -lgmp -o /tmp/test_bfv_private_taint + valgrind --tool=memcheck --error-exitcode=1 --track-origins=yes /tmp/test_bfv_private_taint + if valgrind --tool=memcheck --error-exitcode=1 /tmp/test_bfv_private_taint --negative-control; then + exit 1 + fi + done + + - name: BFV Python binding sanitizers + run: | + bfv_py_include=$(.venv/bin/python -c 'import sysconfig; print(sysconfig.get_paths()["include"])') + g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer -fPIC -shared -I"$bfv_py_include" src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp -lgmpxx -lgmp -o /tmp/bfv_sanitized.so + g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer -fPIC -shared -I"$bfv_py_include" src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp -lgmpxx -lgmp -o /tmp/bfv_private_sanitized.so + bfv_asan_runtime=$(g++ -print-file-name=libasan.so) + bfv_cpp_runtime=$(g++ -print-file-name=libstdc++.so) + ASAN_OPTIONS=detect_leaks=0 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 LD_PRELOAD="$bfv_asan_runtime:$bfv_cpp_runtime" .venv/bin/python tests/x_vec/native/run_binding_sanitizers.py /tmp/bfv_sanitized.so /tmp/bfv_private_sanitized.so + # ── Integration tests (require XTrace API credentials) ────────── # Triggers: # - PRs: only when explicitly labeled `run-integration` (avoids quota diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e35be6..bf1e4c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,14 +4,28 @@ All notable changes to this project will be documented in this file. ### Changed +- Native `BFVClient` server evaluation now defaults to cached GMP key switching: gadget products are accumulated and folded before unpacking, with fewer coefficient checks in Python. `server_backend="reference"` retains the original arithmetic. Parameters, ciphertexts and serialized keys remain compatible. A paired server benchmark checks identical ciphertext outputs and separates first-query from warm-query timings. - **Device selection moved from the `DEVICE` environment variable to a `device=` constructor keyword** on `PaillierClient`, `PaillierLookupClient`, `ExecutionContext.create`, `ExecutionContext.load_from_disk`, and `ExecutionContext.load_from_remote`. The new default is `device="auto"`: clients probe for the GPU extension at construction time and fall back to CPU silently when no GPU is present. If a GPU is detected on the host (`/dev/nvidia0`, `nvidia-smi`, or `CUDA_VISIBLE_DEVICES`) but the extension fails to load, a warning is emitted instead of a silent CPU fallback so misconfigured GPU hosts don't quietly run on CPU. `device="gpu"` raises if the extension is unavailable; `device="cpu"` skips probing entirely. - The `DEVICE` environment variable is no longer read. Existing callers must migrate to the `device=` kwarg. ### Added +- Optional AWS Nitro `BFVAttestedClient`/`BFVAttestedServer` sessions: signed owner requests, exact image measurement pins, fresh attestation, and enclave response receipts checked before private decryption. The enclave evaluates once; the BFV secret stays on the owner. Includes the `bfv-nitro` extra, bounded relay/service, local image build records and an opt-in real AWS test. This remains experimental and requires hardware validation and independent review before production use. +- Persistent BFV `server_backend="residue"` arithmetic with a product of 60-bit primes, fused NTTs and exact RNS tensor scaling. Existing GMP, reference and earlier native backends remain available for differential tests; selecting the product modulus requires fresh keys and a new index. +- BFV execution policies, bounded authenticated sessions, protected private exports, and an owner-controlled recomputing verifier. A separate native private decoder provides fixed-work terminal arithmetic with locked/wiped buffers; this is not an end-to-end constant-time guarantee. +- Complete C++ BFV server evaluation through `server_backend="native"`: direct packed ciphertext buffers, prepared masks, native result merging and compaction, lazy transforms, Barrett reduction and certified CRT conversion modulo q with exact fallback. Includes opt-in native phase profiling; existing BFV keys, wire formats and backends remain compatible. The current optional extensions require public ABI version 4 and private ABI version 1. + +- Optional native BFV `server_backend="rns"`, with independently implemented C++ RNS/NTT polynomial arithmetic, cached transformed evaluation keys, exact GMP reconstruction, and a native Hamming tile circuit. Preserves native BFV keys, ciphertext outputs and wire format. Includes native-boundary tests, sanitizer oracles, optional SEAL benchmarks and correctly tagged binary wheels. The previous GMP and reference backends remain available. +- Native GMP BFV primitives in `crypto/encryption/bfv.py` and a `BFVClient` in `crypto/bfv_client.py`, with CRT batching, ciphertext multiplication/relinearization, public-key Hamming evaluation, packed distance responses, and terminal modulus compaction. Includes CPU benchmarks and tests against plaintext algebra and an optional SEAL oracle. This is an experimental local implementation; the existing Paillier clients and SEAL research example are preserved. +- Offline BFV packed Hamming experiment under `experiments/bfv`, with an isolated public-key server evaluator, correctness tests, a comparison against an exported Paillier Git revision, and a research report. The production Paillier path and dependencies are unchanged. - `xtrace_sdk.x_vec.crypto.device.resolve_device` — shared device-resolution helper used by both Paillier clients and intended for future homomorphic clients with optional GPU backends. - `test_cross_device_interop` — parametrised test covering CPU↔GPU portability for both `PaillierClient` and `PaillierLookupClient`. Verifies that contexts saved on one device load correctly on the other (hash equality), and that ciphertexts produced on either backend round-trip correctly through the other for both encryption and server-side homomorphic add. +### Security + +- BFV regression tests demonstrate secret-key recovery from malicious ciphertexts and observable post-decryption accept/reject behavior, even with private result checks. The guarded and Nitro clients require approval before decryption. Attestation authenticates approved execution under the TEE's assumptions; raw BFV remains malleable and is not given a general chosen-ciphertext security claim. +- Nitro rejects malformed certificate versions and duplicate extensions through the bounded protocol error path. Staging PRs now run quality checks, SEAL reference tests and strict documentation builds; the native sanitizer harness also supports the CFFI dependency installed by the Nitro extra. + ## [0.2.0] - 2026-04-18 ### Added diff --git a/benchmarks/bfv_client_matrix.py b/benchmarks/bfv_client_matrix.py new file mode 100644 index 0000000..3d33c1a --- /dev/null +++ b/benchmarks/bfv_client_matrix.py @@ -0,0 +1,347 @@ +#!/usr/bin/env python3 +"""CPU BFV/Paillier Hamming comparison with separate setup, compute and wire sizes. + +Run from the checkout: .venv/bin/python benchmarks/bfv_client_matrix.py --help +Only synthetic plaintexts are seeded. Encryption always uses scheme randomness. +No network, CUDA, SEAL or XTrace service is required. +""" + +from __future__ import annotations + +import argparse +import gc +import hashlib +import json +import platform +import random +import statistics +import subprocess +import sys +import time +from collections.abc import Callable, Sequence +from datetime import UTC, datetime +from pathlib import Path +from typing import Any, TypeVar + +import gmpy2 +import msgpack + +REPO_ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(REPO_ROOT / "src")) + +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient # noqa: E402 +from xtrace_sdk.x_vec.crypto.encryption.bfv import BFV, MAX_PUBLIC_KEY_CHARS # noqa: E402 +from xtrace_sdk.x_vec.crypto.encryption.paillier import Paillier # noqa: E402 +from xtrace_sdk.x_vec.crypto.paillier_client import PaillierClient # noqa: E402 +from xtrace_sdk.x_vec.crypto.paillier_lookup_client import PaillierLookupClient # noqa: E402 + +T = TypeVar("T") +VARIANTS = ("bfv-packed", "bfv-individual", "paillier-cpu", "paillier-lookup-cpu") + + +def time_call(fn: Callable[[], T]) -> tuple[float, T]: + gc.collect() + start = time.perf_counter() + result = fn() + return time.perf_counter() - start, result + + +def packet(ciphers: Sequence[Sequence[int]], count: int) -> bytes: + """Same MessagePack framing for both schemes; big integers become LE bytes. + + Candidate IDs are implicit positions, so no IDs, content, HTTP or TLS bytes + are included. Native BFV's per-ciphertext header remains in the measurement. + """ + return msgpack.packb( + { + "vector_count": count, + "ciphertexts": [ + [int(v).to_bytes(max(1, (int(v).bit_length() + 7) // 8), "little") for v in row] + for row in ciphers + ], + }, + use_bin_type=True, + ) + + +def unpack_packet(payload: bytes) -> list[list[int]]: + data = msgpack.unpackb(payload, raw=False) + return [[int.from_bytes(v, "little") for v in row] for row in data["ciphertexts"]] + + +def make_data( + count: int, dimension: int, seed: int +) -> tuple[list[list[int]], list[int], list[int]]: + rng = random.Random(seed) + query = [rng.getrandbits(1) for _ in range(dimension)] + vectors = [[rng.getrandbits(1) for _ in range(dimension)] for _ in range(count)] + vectors[0] = query[:] + if count > 1: + vectors[1] = [1 - bit for bit in query] + if count > 2: + vectors[-1] = query[:] + expected = [sum(a != b for a, b in zip(v, query, strict=True)) for v in vectors] + return vectors, query, expected + + +def benchmark( + variant: str, + args: argparse.Namespace, + vectors: list[list[int]], + query: list[int], + expected: list[int], +) -> dict[str, Any]: + n = len(vectors) + timings: dict[str, float] = {} + print(f"{variant}: key generation and public server setup", file=sys.stderr, flush=True) + native = variant.startswith("bfv-") + client: Any + if native: + timings["key_generation_s"], client = time_call( + lambda: BFVClient( + embed_len=args.embed_len, + poly_modulus_degree=args.poly_modulus_degree, + plain_modulus=args.plain_modulus, + coeff_modulus_bits=args.coeff_modulus_bits, + decomposition_bits=args.decomposition_bits, + response_modulus_bits=args.response_modulus_bits, + rns_modulus=args.rns_modulus, + device="cpu", + server_backend=args.server_backend, + ) + ) + elif variant == "paillier-cpu": + timings["key_generation_s"], client = time_call( + lambda: PaillierClient(args.embed_len, args.key_len, device="cpu") + ) + else: + timings["key_generation_s"], client = time_call( + lambda: PaillierLookupClient(args.embed_len, args.key_len, args.alpha_len, device="cpu") + ) + timings["public_key_export_s"], public_json = time_call(client.stringify_pk) + public_key_bytes = len(public_json.encode("utf-8")) + config = json.loads(client.stringify_config()) + if native: + + def load_server() -> BFVClient: + result = BFVClient(skip_key_gen=True, server_backend=args.server_backend) + result.load_config(config) + result.load_stringified_keys( + public_json, max_public_key_chars=args.max_public_key_chars + ) + assert result.keys is None + return result + + timings["public_key_import_s"], server = time_call(load_server) + else: + # Paillier server multiplication needs n^2 only; retain a public dict, + # not a Paillier client containing the decryption key. + timings["public_key_import_s"], server_pk = time_call(lambda: json.loads(public_json)) + modulus = gmpy2.mpz(server_pk["n_squared"]) + public_json = "" + + packed = variant == "bfv-packed" + print(f"{variant}: encrypting {n} vectors and query", file=sys.stderr, flush=True) + timings["index_encrypt_s"], index = time_call( + lambda: client.encrypt_vec_packed(vectors) if packed else client.encrypt_vec_batch(vectors) + ) + timings["query_encrypt_s"], encrypted_query = time_call(lambda: client.encrypt_vec_one(query)) + timings["index_serialize_s"], index_wire = time_call(lambda: packet(index, n)) + timings["query_serialize_s"], query_wire = time_call(lambda: packet([encrypted_query], 1)) + # Feed actual deserialized packets to the evaluator, and do the same for the + # response. Byte totals are measured lengths, not object-size estimates. + timings["index_deserialize_s"], server_index = time_call(lambda: unpack_packet(index_wire)) + timings["query_deserialize_s"], server_query = time_call(lambda: unpack_packet(query_wire)[0]) + + def evaluate() -> list[list[int]]: + if packed: + return server.encode_hamming_server_packed( + server_query, server_index, n, compact=not args.no_compact + ) + if native: + return [ + server.encode_hamming_server(server_query, ct, compact=not args.no_compact) + for ct in server_index + ] + return [ + [int(gmpy2.mpz(a) * b % modulus) for a, b in zip(server_query, row, strict=True)] + for row in server_index + ] + + print(f"{variant}: public server evaluation", file=sys.stderr, flush=True) + timings["server_compute_s"], response = time_call(evaluate) + timings["response_serialize_s"], response_wire = time_call(lambda: packet(response, n)) + timings["response_deserialize_s"], client_response = time_call( + lambda: unpack_packet(response_wire) + ) + timings["client_decode_s"], distances = time_call( + lambda: ( + client.decode_hamming_client_packed(client_response, n) + if packed + else client.decode_hamming_client_batch(client_response) + ) + ) + if distances != expected: + raise AssertionError( + f"{variant}: decrypted distances differ from the plaintext Hamming oracle" + ) + top3 = sorted(range(n), key=lambda i: (distances[i], i))[:3] + assert top3 == sorted(range(n), key=lambda i: (expected[i], i))[:3] + budgets = [] + if native: + budgets = [ + BFV.noise_budget(BFV.ciphertext_from_ints(ct, client.public_key), client.keys) + for ct in client_response + ] + sizes = { + "encrypted_index_bytes": len(index_wire), + "public_keys_bytes": public_key_bytes, + "query_bytes": len(query_wire), + "response_bytes": len(response_wire), + "query_plus_response_bytes": len(query_wire) + len(response_wire), + } + print( + f"{variant}: verified {n} distances; response {len(response_wire):,} bytes", + file=sys.stderr, + flush=True, + ) + return { + "variant": variant, + "server_backend": args.server_backend if native else None, + "vectors": n, + "embed_len": args.embed_len, + "config": config, + "compact": native and not args.no_compact, + "timings": timings, + "sizes": sizes, + "index_ciphertexts": len(index), + "response_ciphertexts": len(response), + "minimum_response_noise_budget_bits": min(budgets) if budgets else None, + "top3": [{"index": i, "distance": distances[i]} for i in top3], + "correct": True, + } + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--num-vectors", type=int, default=256) + parser.add_argument("--embed-len", type=int, default=512) + parser.add_argument( + "--key-len", + type=int, + default=1024, + help="Paillier prime bit length, as in the existing clients", + ) + parser.add_argument("--alpha-len", type=int, default=50) + parser.add_argument("--poly-modulus-degree", type=int, default=8192) + parser.add_argument("--plain-modulus", type=int, default=65537) + parser.add_argument("--coeff-modulus-bits", type=int, default=180) + parser.add_argument("--decomposition-bits", type=int, default=30) + parser.add_argument("--response-modulus-bits", type=int, default=50) + parser.add_argument( + "--max-public-key-chars", + type=int, + default=MAX_PUBLIC_KEY_CHARS, + help="Local JSON import bound; use 268435456 for the N=16384 review profile", + ) + parser.add_argument("--no-compact", action="store_true") + parser.add_argument( + "--server-backend", + choices=("optimized", "reference", "rns", "native", "residue"), + default="optimized", + ) + parser.add_argument("--seed", type=int, default=1337) + parser.add_argument( + "--repeats", + type=int, + default=1, + help="Independent runs; each includes fresh key generation", + ) + parser.add_argument( + "--variants", + default="bfv-packed,paillier-cpu", + help="Comma-separated: " + ", ".join(VARIANTS), + ) + parser.add_argument("--json-out", type=Path) + parser.add_argument( + "--rns-modulus", + action="store_true", + help="Use fresh BFV keys with a product of 60-bit NTT primes", + ) + args = parser.parse_args() + if args.server_backend == "residue" and not args.rns_modulus: + parser.error("residue requires --rns-modulus") + variants = [v.strip() for v in args.variants.split(",")] + if ( + args.num_vectors < 1 + or args.embed_len < 1 + or args.repeats < 1 + or args.max_public_key_chars < 1 + ): + parser.error("num-vectors, embed-len, repeats and max-public-key-chars must be positive") + if any(v not in VARIANTS for v in variants): + parser.error("unknown variant") + vectors, query, expected = make_data(args.num_vectors, args.embed_len, args.seed) + revision = subprocess.run( + ["git", "rev-parse", "HEAD"], cwd=REPO_ROOT, capture_output=True, text=True, check=True + ).stdout.strip() + dirty = bool( + subprocess.run( + ["git", "status", "--porcelain"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + ) + output: dict[str, Any] = { + "environment": { + "utc": datetime.now(UTC).isoformat(), + "python": platform.python_version(), + "platform": platform.platform(), + "gmpy2": gmpy2.version(), + "gmp": gmpy2.mp_version(), + "revision": revision, + "working_tree_dirty": dirty, + }, + "command": sys.argv, + "measurement_notes": "CPU synthetic experiment; no equivalent-security claim. Keys are public JSON including BFV evaluation keys. Index/query/response use common MessagePack framing with LE integer bytes and implicit IDs. No network/HTTP/TLS, secret-key, plaintext-content or AES-content bytes. Timings are separate from serialization; no concurrent test/benchmark processes intended.", + "seed": args.seed, + "runs": [], + "source_sha256": { + str(path.relative_to(REPO_ROOT)): hashlib.sha256(path.read_bytes()).hexdigest() + for path in [ + Path(__file__).resolve(), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto").glob("*client.py"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption").glob("*.py"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.h"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.cpp"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.so"), + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile", + REPO_ROOT / "src/xtrace_sdk/x_vec/utils/xtrace_types.py", + ] + }, + } + for repeat in range(args.repeats): + for variant in variants: + row = benchmark(variant, args, vectors, query, expected) + row["repeat"] = repeat + output["runs"].append(row) + gc.collect() + output["median_timings"] = { + v: { + metric: statistics.median( + row["timings"][metric] for row in output["runs"] if row["variant"] == v + ) + for metric in output["runs"][0]["timings"] + } + for v in variants + } + print(json.dumps(output, indent=2)) + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text(json.dumps(output, indent=2) + "\n") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/bfv_guarded.py b/benchmarks/bfv_guarded.py new file mode 100644 index 0000000..71b772e --- /dev/null +++ b/benchmarks/bfv_guarded.py @@ -0,0 +1,237 @@ +#!/usr/bin/env python3 +"""Measure public recomputation and private decoding on identical BFV ciphertexts. + +Run the original and review profiles separately, without concurrent CPU work. +Only synthetic plaintext is seeded. JSON contains measurements and public +parameters, never keys, ciphertexts, signing seeds or private fingerprints. +""" + +from __future__ import annotations + +import argparse +import hashlib +import importlib.metadata +import json +import platform +import resource +import secrets +import statistics +import subprocess +import sys +from dataclasses import replace +from datetime import UTC, datetime +from functools import partial +from pathlib import Path +from typing import Any + +import gmpy2 + +from bfv_client_matrix import REPO_ROOT, make_data, time_call +from xtrace_sdk.x_vec.crypto.bfv_assurance import bfv_review_policy, hamming_noise_bound +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient +from xtrace_sdk.x_vec.crypto.bfv_guarded_client import ( + BFVGuardedClient, + BFVPublicVerifier, + bfv_verifier_public_key, +) +from xtrace_sdk.x_vec.crypto.bfv_security import ( + BFVExecutionPolicy, + _authenticated_body, + _decode_ciphertexts, + _unpack, +) +from xtrace_sdk.x_vec.crypto.bfv_verified_client import BFVVerifiedServer + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--profile", choices=("original", "review"), default="review") + parser.add_argument("--num-vectors", type=int, default=1024) + parser.add_argument("--embed-len", type=int, default=512) + parser.add_argument("--seed", type=int, default=1337) + parser.add_argument("--repeats", type=int, default=4, help="One cold and three warm trials") + parser.add_argument("--json-out", type=Path) + args = parser.parse_args() + if min(args.num_vectors, args.embed_len) < 1 or args.repeats < 2: + parser.error("positive dimensions and at least two repeats are required") + policy = replace( + BFVExecutionPolicy() if args.profile == "original" else bfv_review_policy(), + embed_len=args.embed_len, + ) + if args.num_vectors > policy.max_vectors or args.repeats > policy.max_queries: + parser.error("workload exceeds the local session policy") + vectors, query, expected = make_data(args.num_vectors, args.embed_len, args.seed) + setup_times: dict[str, float] = {} + timings: dict[str, list[float]] = { + name: [] + for name in ( + "server", + "verifier", + "python_raw_client", + "native_raw_client", + "guarded_client", + ) + } + print(f"Preparing {args.profile} profile", file=sys.stderr, flush=True) + setup_times["key_generation_s"], raw = time_call(lambda: BFVClient(**policy.config())) + auth, signing_seed = secrets.token_bytes(32), secrets.token_bytes(32) + setup_times["client_key_snapshot_native_import_s"], client = time_call( + lambda: BFVGuardedClient(raw, auth, bfv_verifier_public_key(signing_seed), policy=policy) + ) + setup_times["encrypt_index_summary_export_auth_s"], setup = time_call( + lambda: client.prepare_index(vectors) + ) + setup_times["server_setup_import_s"], server = time_call( + lambda: BFVVerifiedServer(setup, auth, policy=policy) + ) + setup_times["verifier_setup_import_s"], verifier = time_call( + lambda: BFVPublicVerifier( + setup, auth, signing_seed, expected_setup_digest=client.setup_digest, policy=policy + ) + ) + assert server._client.keys is None and verifier._server._client.keys is None + setup_times["server_arithmetic_context_s"], _ = time_call(server._client._evaluator) + setup_times["verifier_arithmetic_context_s"], _ = time_call(verifier._server._client._evaluator) + native = client._session._private_decoder + assert native is not None + pk = raw._pk() + count = ( + len(vectors) + policy.params.poly_modulus_degree - 1 + ) // policy.params.poly_modulus_degree + query_times, runs = [], [] + request = response = receipt = b"" + + def raw_finish(wire: bytes, use_native: bool) -> list[int]: + # Benchmark-only direct private primitive access: applications must use + # finish_query(response, receipt). Both reference timers include parsing. + cts = _decode_ciphertexts(wire, pk, count, policy.max_response_bytes) + if use_native: + return native.decode_packed(cts, len(vectors)) + return raw.decode_hamming_client_packed(cts, len(vectors)) + + for repeat in range(args.repeats): + elapsed, request = time_call(lambda: client.begin_query(query)) + query_times.append(elapsed) + elapsed, response = time_call(partial(server.search, request)) + timings["server"].append(elapsed) + print(f"server: {elapsed:.6f} s", file=sys.stderr, flush=True) + elapsed, receipt = time_call(partial(verifier.approve, request, response)) + timings["verifier"].append(elapsed) + print(f"verifier: {elapsed:.6f} s", file=sys.stderr, flush=True) + wire = _unpack( + _authenticated_body(response, auth, 3, policy.max_response_bytes), + limit=policy.max_response_bytes, + )[3] + variants = ("python_raw_client", "native_raw_client", "guarded_client") + for name in variants if repeat % 2 == 0 else reversed(variants): + elapsed, distances = time_call( + partial(client.finish_query, response, receipt) + if name == "guarded_client" + else partial(raw_finish, wire, name == "native_raw_client") + ) + assert distances == expected, "Incorrect Hamming distances" + timings[name].append(elapsed) + runs.append({"repeat": repeat, **{name: samples[-1] for name, samples in timings.items()}}) + setup_fields = _unpack( + _authenticated_body(setup, auth, 1, policy.max_setup_bytes), + limit=policy.max_setup_bytes, + array_limit=policy.max_vectors, + ) + crypto = REPO_ROOT / "src/xtrace_sdk/x_vec/crypto" + sources = [ + Path(__file__).resolve(), + REPO_ROOT / "benchmarks/bfv_client_matrix.py", + *crypto.glob("bfv*.py"), + *(crypto / "encryption").glob("bfv*.py"), + *(crypto / "bfv_cpu_ext").glob("*.cpp"), + *(crypto / "bfv_cpu_ext").glob("*.h"), + *(crypto / "bfv_cpu_ext").glob("*.so"), + crypto / "bfv_cpu_ext/Makefile", + REPO_ROOT / "src/xtrace_sdk/x_vec/utils/xtrace_types.py", + ] + output: dict[str, Any] = { + "environment": { + "utc": datetime.now(UTC).isoformat(), + "python": platform.python_version(), + "platform": platform.platform(), + "cpu": next( + ( + line.split(":", 1)[1].strip() + for line in Path("/proc/cpuinfo").read_text().splitlines() + if line.startswith("model name") + ), + "unknown", + ), + "gmpy2": gmpy2.version(), + "gmp": gmpy2.mp_version(), + "pycryptodome": importlib.metadata.version("pycryptodome"), + "msgpack": importlib.metadata.version("msgpack"), + "compiler": subprocess.run( + ["c++", "--version"], capture_output=True, text=True, check=True + ).stdout.splitlines()[0], + "revision": subprocess.run( + ["git", "rev-parse", "HEAD"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip(), + "working_tree_dirty": bool( + subprocess.run( + ["git", "status", "--porcelain"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + ), + }, + "command": sys.argv, + "profile": args.profile, + "config": policy.config(), + "vectors": len(vectors), + "seed": args.seed, + "measurement_notes": "Within each profile all variants use identical keys and evaluated ciphertexts. Server and owner-controlled public-only verifier have independently imported setups and separate residue arithmetic contexts. Server search and verifier approval include authentication, MessagePack and arithmetic; setup import, arithmetic-context creation, query encryption and private decoding are separate. Approval independently recomputes the full result and compares exact bytes before Ed25519 signing. Server then verifier run sequentially; no network or overlap claimed. One cold trial followed by three warm trials by default. Raw Python/native client timers include ciphertext-array parsing and all distance decoding; guarded includes receipt, authentication, replay and private result checks, with alternating client order. No security-level or end-to-end constant-time claim.", + "setup_timings": setup_times, + "runs": runs, + "summary": { + name: { + "first_s": samples[0], + "warm_samples_s": samples[1:], + "warm_median_s": statistics.median(samples[1:]), + } + for name, samples in timings.items() + }, + "query_encrypt_and_bind_samples_s": query_times, + "sizes": { + "public_keys_bytes_per_evaluator": len(setup_fields[2]), + "encrypted_index_bytes_per_evaluator": len(setup_fields[5]), + "authenticated_setup_bytes_per_evaluator": len(setup), + "query_bytes": len(request), + "response_bytes": len(response), + "receipt_bytes": len(receipt), + "client_response_and_receipt_bytes": len(response) + len(receipt), + "client_query_response_receipt_bytes": len(request) + len(response) + len(receipt), + "remote_verifier_additional_query_response_input_bytes": len(request) + len(response), + "private_verification_payload_bytes": client.verification_state_bytes, + "native_locked_key_bytes": 16 * policy.params.poly_modulus_degree, + "native_locked_scratch_bytes_per_active_decode": 24 * policy.params.poly_modulus_degree, + }, + "whole_process_peak_rss_kib_linux": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss, + "noise_bound": hamming_noise_bound(policy, policy.max_vectors).as_dict(), + "identical_ciphertexts_within_profile": True, + "all_distances_correct": True, + "source_sha256": { + str(p.relative_to(REPO_ROOT)): hashlib.sha256(p.read_bytes()).hexdigest() + for p in sources + }, + } + encoded = json.dumps(output, indent=2) + print(encoded) + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text(encoded + "\n") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/bfv_nitro.py b/benchmarks/bfv_nitro.py new file mode 100644 index 0000000..765d5ff --- /dev/null +++ b/benchmarks/bfv_nitro.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python3 +"""Compare one Nitro-protocol evaluation with the same-ciphertext BFV baseline. + +--mode local uses a SYNTHETIC TEST CA and measures protocol overhead only. It +cannot measure Nitro isolation, NSM or EC2 overhead. --mode aws requires an +empty real enclave plus owner-trusted EIF pins; its server timer includes RPC. +Only generated synthetic plaintext is seeded; no keys or ciphertexts are logged. +""" + +from __future__ import annotations + +import argparse +import hashlib +import importlib.metadata +import json +import platform +import resource +import secrets +import statistics +import subprocess +import sys +from contextlib import ExitStack +from datetime import UTC, datetime +from pathlib import Path +from unittest.mock import patch + +from bfv_client_matrix import REPO_ROOT, make_data, time_call +sys.path.insert(0, str(REPO_ROOT)) +from xtrace_sdk.x_vec.crypto.bfv_assurance import bfv_review_policy +from xtrace_sdk.x_vec.crypto.bfv_attested_client import BFVAttestedClient, BFVAttestedServer +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient +from xtrace_sdk.x_vec.crypto.bfv_verified_client import BFVVerifiedServer +from xtrace_sdk.x_vec.crypto.bfv_nitro import NitroAttestationPolicy + + +def main(): + """Record explicit evidence provenance, raw samples, sizes and exact equality.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--mode", choices=("local", "aws"), required=True) + parser.add_argument("--pins", type=Path) + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=9000) + parser.add_argument("--num-vectors", type=int, default=1024) + parser.add_argument("--repeats", type=int, default=4) + parser.add_argument("--json-out", type=Path) + args = parser.parse_args() + policy = bfv_review_policy() + if not 1 <= args.num_vectors <= policy.max_vectors or not 2 <= args.repeats <= 128: + parser.error("Use a bounded vector count and 2..128 repeats") + if args.mode == "aws" and args.pins is None: + parser.error("AWS mode requires owner-trusted --pins") + with ExitStack() as stack: + if args.mode == "local": + # This benchmark-only trust substitution is intentionally absent + # from the SDK's public API and the measured service image. + from tests.x_vec.nitro_fixtures import PCRS, SyntheticNitro + from xtrace_sdk.x_vec.crypto import bfv_nitro + issuer = SyntheticNitro() + stack.enter_context(patch.object(bfv_nitro, "_AWS_ROOT_SHA256", issuer.root_digest)) + pins = NitroAttestationPolicy(PCRS) + print("LOCAL SYNTHETIC EVIDENCE: protocol measurements, no TEE hardware", file=sys.stderr, flush=True) + else: + from experiments.bfv_nitro.client import load_measurements + pins = load_measurements(args.pins) + vectors, query, expected = make_data(args.num_vectors, 512, 1337) + setup_times = {} + setup_times["key_generation_s"], raw = time_call(lambda: BFVClient(**policy.config())) + auth = secrets.token_bytes(32) + client = BFVAttestedClient(raw, auth, pins, index_epoch=1, policy=policy) + setup_times["index_encrypt_export_s"], setup = time_call(lambda: client.prepare_index(vectors)) + setup_times["registration_encode_s"], registration = time_call(lambda: client.registration_packet(setup)) + setup_times["baseline_import_s"], baseline = time_call(lambda: BFVVerifiedServer(setup, auth, policy=policy)) + if args.mode == "local": + setup_times["protocol_import_s"], endpoint = time_call(lambda: BFVAttestedServer.from_registration(registration, issuer, policy=policy)) + else: + from experiments.bfv_nitro.transport import NitroRemote + endpoint = NitroRemote(args.host, args.port, policy=policy) + setup_times["remote_registration_s"], _ = time_call(lambda: endpoint.register(registration)) + enrollment = [] + for _ in range(args.repeats): + hello = client.begin_attestation() + issue_s, (document, proof) = time_call(lambda: endpoint.attest(hello)) + verify_s, _ = time_call(lambda: client.accept_attestation(document, proof)) + enrollment.append({"issue_or_rpc_s": issue_s, "client_verify_s": verify_s, "document_bytes": len(document), "possession_bytes": len(proof), "challenge_bytes": len(hello)}) + runs = [] + for repeat in range(args.repeats): + encrypt_s, request = time_call(lambda: client.begin_query(query)) + baseline_call = lambda: baseline.search(request[38:-64]) + protocol_call = lambda: endpoint.search(request) + # Alternate order to reduce a systematic warm-cache advantage. + if repeat % 2: + protocol_s, (response, receipt) = time_call(protocol_call) + baseline_s, reference = time_call(baseline_call) + else: + baseline_s, reference = time_call(baseline_call) + protocol_s, (response, receipt) = time_call(protocol_call) + if response != reference: + raise RuntimeError("Nitro-protocol output differs from exact BFV baseline") + finish_s, distances = time_call(lambda: client.finish_query(response, receipt)) + if distances != expected: + raise RuntimeError("Incorrect Hamming distances") + runs.append({"repeat": repeat, "query_encrypt_sign_s": encrypt_s, "baseline_local_search_s": baseline_s, "protocol_search_or_rpc_s": protocol_s, "client_finish_s": finish_s}) + print(f"trial {repeat}: baseline {baseline_s:.6f}s, protocol/RPC {protocol_s:.6f}s, client {finish_s:.6f}s", file=sys.stderr, flush=True) + summaries = { + key: {"first_s": runs[0][key], "warm_median_s": statistics.median(r[key] for r in runs[1:]), "warm_samples_s": [r[key] for r in runs[1:]]} + for key in runs[0] if key != "repeat" + } + paths = [Path(__file__), *REPO_ROOT.glob("src/xtrace_sdk/x_vec/crypto/bfv*.py"), *REPO_ROOT.glob("src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/*.so"), *REPO_ROOT.glob("experiments/bfv_nitro/*.py")] + output = { + "mode": args.mode, + "hardware_attestation": args.mode == "aws", + "measurement_notes": "All variants evaluate identical ciphertexts; exactly one evaluation produces a protocol response. Local mode substitutes a test CA and includes real certificate, owner-signature and receipt checks, but no NSM, vsock, hardware isolation or EC2. AWS protocol timers include network/relay; the reference runs locally and ratios are meaningful only on matched hardware. One cold and remaining warm trials; no tail-latency claim. No production-security certification.", + "environment": {"utc": datetime.now(UTC).isoformat(), "python": platform.python_version(), "platform": platform.platform(), "cpu": next((line.split(':', 1)[1].strip() for line in Path('/proc/cpuinfo').read_text().splitlines() if line.startswith('model name')), 'unknown'), "versions": {name: importlib.metadata.version(name) for name in ("gmpy2", "pycryptodome", "cbor2", "pyOpenSSL", "cryptography")}, "revision": subprocess.run(["git", "rev-parse", "HEAD"], cwd=REPO_ROOT, check=True, text=True, capture_output=True).stdout.strip()}, + "config": policy.config(), "vectors": len(vectors), "setup_timings": setup_times, + "enrollment": enrollment, "runs": runs, "summary": summaries, + "sizes": {"setup_bytes": len(setup), "registration_bytes": len(registration), "query_bytes": len(request), "response_bytes": len(response), "receipt_bytes": len(receipt), "query_response_receipt_bytes": len(request) + len(response) + len(receipt)}, + "whole_process_peak_rss_kib_linux": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss, + "identical_ciphertexts": True, "all_distances_correct": True, + "source_sha256": {str(path.resolve().relative_to(REPO_ROOT)): hashlib.sha256(path.read_bytes()).hexdigest() for path in sorted(set(paths))}, + } + text = json.dumps(output, indent=2) + "\n" + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text(text) + print(text, end="") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/bfv_parameter_audit.py b/benchmarks/bfv_parameter_audit.py new file mode 100644 index 0000000..35f1506 --- /dev/null +++ b/benchmarks/bfv_parameter_audit.py @@ -0,0 +1,186 @@ +#!/usr/bin/env python3 +"""Export the actual BFV profile and run a pinned, external lattice estimator. + +Use the SDK Python for `manifest`, and a Sage Python for `estimate`. +This reports heuristic attack costs, not a certificate for BFV or its protocol. +It never generates or exports cryptographic keys. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import platform +import subprocess +import sys +import time +from datetime import UTC, datetime +from functools import partial +from pathlib import Path +from typing import Any + +REPO_ROOT = Path(__file__).resolve().parents[1] +ESTIMATOR_REVISION = "53da5982597709ba0fdf94ea37a84d822310fd84" + + +def revision(path: Path) -> str: + return subprocess.run( + ["git", "rev-parse", "HEAD"], cwd=path, check=True, capture_output=True, text=True + ).stdout.strip() + + +def manifest(profile: str) -> dict[str, Any]: + sys.path.insert(0, str(REPO_ROOT / "src")) + from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient + from xtrace_sdk.x_vec.crypto.bfv_assurance import bfv_review_policy, hamming_noise_bound + from xtrace_sdk.x_vec.crypto.bfv_security import BFVExecutionPolicy + from xtrace_sdk.x_vec.crypto.encryption.bfv import _parameter_modulus, _rns_coefficient_primes + + policy = BFVExecutionPolicy() if profile == "original" else bfv_review_policy() + client = BFVClient(**policy.config(), skip_key_gen=True) + params = policy.params + n = params.poly_modulus_degree + rotations = {pow(3, s % (n // 2), 2 * n) for s in client._rotation_steps()} - {1} + digits = ( + params.coeff_modulus_bits + params.decomposition_bits - 1 + ) // params.decomposition_bits + sources = [ + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py", + ] + return { + "schema": 1, + "profile": profile, + "config": policy.config(), + "q": str(_parameter_modulus(params)), + "q_primes": _rns_coefficient_primes(n, params.coeff_modulus_bits), + "secret": {"distribution": "uniform", "inclusive_bounds": [-1, 1]}, + "error": { + "distribution": "centered_binomial", + "eta": params.error_eta, + "variance": params.error_eta / 2, + }, + "ephemeral_u": {"distribution": "uniform", "inclusive_bounds": [-1, 1]}, + "evaluation_keys": { + "galois_exponents": sorted(rotations), + "digits_per_key": digits, + "key_switch_pairs": (1 + len(rotations)) * digits, + "note": "Encryptions of gadget-scaled s^2 and automorphic secrets. Ordinary LWE estimates do not justify these circular/key-dependent-message assumptions.", + }, + "sdk_revision": revision(REPO_ROOT), + "worst_supported_hamming_bound": hamming_noise_bound(policy, policy.max_vectors).as_dict(), + "source_sha256": { + p: hashlib.sha256((REPO_ROOT / p).read_bytes()).hexdigest() for p in sources + }, + } + + +def estimate(args: argparse.Namespace) -> dict[str, Any]: + root = args.estimator.resolve() + actual_revision = revision(root) + if actual_revision != ESTIMATOR_REVISION: + raise ValueError( + f"Expected reviewed estimator revision {ESTIMATOR_REVISION}, got {actual_revision}" + ) + if subprocess.run( + ["git", "diff", "--exit-code", "HEAD"], cwd=root, capture_output=True + ).returncode: + raise ValueError("Estimator tracked sources must match the pinned revision") + data = json.loads(args.manifest.read_text()) + if data["schema"] != 1 or data["secret"] != { + "distribution": "uniform", + "inclusive_bounds": [-1, 1], + }: + raise ValueError("Unsupported parameter manifest") + if data["error"]["distribution"] != "centered_binomial": + raise ValueError("Unsupported error distribution") + sys.path.insert(0, str(root)) + from sage.all import ZZ, log, oo + from sage.env import SAGE_VERSION + from estimator import LWE, ND, RC + from estimator.reduction import ADPS16 + + n, q = data["config"]["poly_modulus_degree"], ZZ(data["q"]) + models = { + "MATZOV-classical": RC.MATZOV, + "ADPS16-classical": ADPS16(), + "ADPS16-quantum": ADPS16(mode="quantum"), + } + attacks = { + "primal_usvp": LWE.primal_usvp, + "primal_bdd": LWE.primal_bdd, + "dual": LWE.dual, + "dual_hybrid": LWE.dual_hybrid, + } + selected_models = args.models.split(",") + selected_attacks = args.attacks.split(",") + if any(m not in models for m in selected_models) or any( + a not in attacks for a in selected_attacks + ): + raise ValueError("Unknown cost model or attack") + runs: list[dict[str, Any]] = [] + output = { + "utc": datetime.now(UTC).isoformat(), + "sage": SAGE_VERSION, + "python": platform.python_version(), + "estimator_revision": actual_revision, + "runner_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(), + "manifest": data, + "command": sys.argv, + "interpretation": "Heuristic generic LWE attack estimates with Gaussian-shape approximation for centered binomial errors. Reports m=N and unbounded independent samples; RLWE structure, evaluation-key assumptions, decryption failures, implementation leakage and protocol attacks are not certified. ADPS16-quantum is a quantum sieving cost sensitivity model, not a complete quantum security analysis. Unlisted attacks are not evaluated.", + "runs": runs, + "complete": False, + } + for sample_label, samples in (("N", n), ("unbounded", oo)): + params = LWE.Parameters( + n=n, q=q, Xs=ND.Uniform(-1, 1), Xe=ND.CenteredBinomial(data["error"]["eta"]), m=samples + ) + for model_name in selected_models: + for attack_name in selected_attacks: + print( + f"Estimating {sample_label} samples / {model_name} / {attack_name}", flush=True + ) + kwargs = {"red_cost_model": models[model_name]} + if attack_name.startswith("primal"): + kwargs["red_shape_model"] = "gsa" + start = time.perf_counter() + cost = attacks[attack_name](params, **kwargs) + record = { + "samples": sample_label, + "model": model_name, + "attack": attack_name, + "seconds": time.perf_counter() - start, + "log2_rop": float(log(cost["rop"], 2)) if cost["rop"] != oo else None, + "cost": {str(k): str(v) for k, v in cost.items()}, + } + runs.append(record) + print(json.dumps(record), flush=True) + args.json_out.write_text(json.dumps(output, indent=2) + "\n") + output["complete"] = True + return output + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="mode", required=True) + make = sub.add_parser("manifest") + make.add_argument("--json-out", type=Path, required=True) + make.add_argument("--profile", choices=("original", "review"), default="original") + run = sub.add_parser("estimate") + run.add_argument("--manifest", type=Path, required=True) + run.add_argument("--estimator", type=Path, required=True) + run.add_argument("--json-out", type=Path, required=True) + run.add_argument("--models", default="MATZOV-classical,ADPS16-classical,ADPS16-quantum") + run.add_argument("--attacks", default="primal_usvp,primal_bdd,dual,dual_hybrid") + args = parser.parse_args() + args.json_out.parent.mkdir(parents=True, exist_ok=True) + output = manifest(args.profile) if args.mode == "manifest" else estimate(args) + args.json_out.write_text(json.dumps(output, indent=2) + "\n") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/bfv_server.py b/benchmarks/bfv_server.py new file mode 100644 index 0000000..fd9a705 --- /dev/null +++ b/benchmarks/bfv_server.py @@ -0,0 +1,366 @@ +#!/usr/bin/env python3 +"""Compare native BFV server backends on identical keys, queries, and index bytes. + +The first search includes lazy evaluation-key preparation. Subsequent searches +reuse the same public-only server. No SEAL, network, GPU or secret server key is +required. Run without concurrent tests/benchmarks for useful timing comparisons. +""" + +from __future__ import annotations + +import argparse +import cProfile +import hashlib +import importlib.metadata +import json +import platform +import statistics +import subprocess +import sys +from datetime import UTC, datetime +from functools import partial +from pathlib import Path +from typing import Any, cast + +import gmpy2 + +from bfv_client_matrix import REPO_ROOT, make_data, packet, time_call, unpack_packet +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient +from xtrace_sdk.x_vec.crypto.encryption.bfv import BFV, _rns_coefficient_primes +from xtrace_sdk.x_vec.crypto.encryption.bfv_evaluator import BFVEvaluator, BFVServerBackend + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--num-vectors", type=int, default=1024) + parser.add_argument("--embed-len", type=int, default=512) + parser.add_argument("--poly-modulus-degree", type=int, default=8192) + parser.add_argument("--plain-modulus", type=int, default=65537) + parser.add_argument("--coeff-modulus-bits", type=int, default=180) + parser.add_argument("--decomposition-bits", type=int, default=30) + parser.add_argument( + "--rns-modulus", + action="store_true", + help="Generate fresh keys with a product of 60-bit NTT primes; required by residue", + ) + parser.add_argument("--response-modulus-bits", type=int, default=50) + parser.add_argument( + "--residue-levels", + default="2", + help="Residue controls: 0=previous kernels, 1=fused NTT, 2=also exact RNS scaling", + ) + parser.add_argument("--seed", type=int, default=1337) + parser.add_argument( + "--repeats", + type=int, + default=3, + help="Searches per backend, including the first cold search", + ) + parser.add_argument("--json-out", type=Path) + parser.add_argument( + "--backends", + default="reference,optimized", + help="Comma-separated native backends: reference,optimized,rns,native,residue", + ) + parser.add_argument( + "--seal", + action="store_true", + help="Also measure the preserved SEAL prototype on the same plaintext workload", + ) + parser.add_argument( + "--profile-dir", + type=Path, + help="Run one extra warm search per backend under cProfile, outside reported timings", + ) + parser.add_argument( + "--native-profile", + action="store_true", + help="Collect exclusive C++ phase timings in additional warm searches, outside reported timings", + ) + args = parser.parse_args() + if args.num_vectors < 1 or args.repeats < 1 or args.embed_len < 1: + parser.error("num-vectors, embed-len and repeats must be positive") + names = args.backends.split(",") + if ( + not names + or len(set(names)) != len(names) + or any(b not in ("reference", "optimized", "rns", "native", "residue") for b in names) + ): + parser.error("backends must be distinct native backend names") + if args.seal and (args.poly_modulus_degree != 8192 or args.plain_modulus != 65537): + parser.error("the SEAL comparison requires N=8192 and t=65537") + if args.native_profile and not any(b in ("rns", "native", "residue") for b in names): + parser.error("native-profile requires the rns, native, or residue backend") + if "residue" in names and not args.rns_modulus: + parser.error("residue requires --rns-modulus (fresh keys and index)") + levels = args.residue_levels.split(",") + if len(set(levels)) != len(levels) or any(level not in ("0", "1", "2") for level in levels): + parser.error("residue-levels must contain distinct values from 0,1,2") + variants: dict[str, tuple[BFVServerBackend, int | None]] = {} + for name in names: + if name == "residue": + for level in levels: + label = "residue" if len(levels) == 1 else f"residue-{level}" + variants[label] = ("residue", int(level)) + else: + variants[name] = (cast(BFVServerBackend, name), None) + backends = tuple(variants) + vectors, query, expected = make_data(args.num_vectors, args.embed_len, args.seed) + print("Generating one key set and encrypting the shared input", file=sys.stderr, flush=True) + setup: dict[str, float] = {} + setup["key_generation_s"], client = time_call( + lambda: BFVClient( + embed_len=args.embed_len, + poly_modulus_degree=args.poly_modulus_degree, + plain_modulus=args.plain_modulus, + coeff_modulus_bits=args.coeff_modulus_bits, + decomposition_bits=args.decomposition_bits, + response_modulus_bits=args.response_modulus_bits, + rns_modulus=args.rns_modulus, + device="cpu", + ) + ) + setup["index_encrypt_s"], index = time_call(lambda: client.encrypt_vec_packed(vectors)) + setup["query_encrypt_s"], encrypted_query = time_call(lambda: client.encrypt_vec_one(query)) + index_wire, query_wire = packet(index, len(vectors)), packet([encrypted_query], 1) + server_index, server_query = unpack_packet(index_wire), unpack_packet(query_wire)[0] + setup["public_key_export_s"], public_json = time_call(client.stringify_pk) + config = json.loads(client.stringify_config()) + servers = {} + for label, (actual_backend, level) in variants.items(): + server = BFVClient(skip_key_gen=True, server_backend=actual_backend, **config) + setup[f"{label}_public_key_import_s"], _ = time_call( + partial(server.load_stringified_keys, public_json) + ) + setup[f"{label}_arithmetic_context_s"], server._server_evaluator = time_call( + partial(BFVEvaluator, server._pk(), actual_backend, kernel_level=level) + ) + server._evaluator_public_key = server._pk() + assert server.keys is None + servers[label] = server + key_bytes = len(public_json.encode()) + del public_json + + def evaluate(backend: str) -> list[list[int]]: + return servers[backend].encode_hamming_server_packed( + server_query, server_index, len(vectors) + ) + + seal_info: dict[str, Any] | None = None + if args.seal: + sys.path.insert(0, str(REPO_ROOT / "experiments/bfv")) + from packed_hamming import BfvClient, BfvServer + + print( + "Preparing the preserved SEAL prototype on the same vectors", + file=sys.stderr, + flush=True, + ) + seal_client = BfvClient(args.embed_len) + seal_bundle = seal_client.server_bundle() + seal_server = BfvServer(seal_bundle) + seal_index, seal_query = ( + seal_client.encrypt_index(vectors), + seal_client.encrypt_query(query), + ) + seal_info = { + "tenseal": importlib.metadata.version("tenseal"), + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + p.bit_count() + for p in seal_client.context.first_context_data().parms().coeff_modulus() + ], + "key_coeff_modulus_bits": [ + p.bit_count() + for p in seal_client.context.key_context_data().parms().coeff_modulus() + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": len(seal_index), + "query_bytes": len(seal_query), + "public_key_bytes": len(seal_bundle), + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + } + + measured_backends = (*backends, "seal") if args.seal else backends + timings: dict[str, list[float]] = {backend: [] for backend in measured_backends} + order = [] + reference_response: list[list[int]] | None = None + for repeat in range(args.repeats): + # Alternate ordering to reduce systematic first/last-run effects. + for backend_name in measured_backends if repeat % 2 == 0 else measured_backends[::-1]: + print( + f"{backend_name}: search {repeat + 1}/{args.repeats}", file=sys.stderr, flush=True + ) + if backend_name == "seal": + elapsed, seal_response = time_call( + partial(seal_server.search, seal_index, seal_query) + ) + assert [d for _, d in seal_client.decrypt_distances(seal_response)] == expected + assert seal_info is not None + seal_info["response_bytes"] = len(seal_response) + seal_info["all_distances_correct"] = True + seal_info["minimum_response_noise_budget_bits"] = min( + seal_client.noise_budgets(seal_response) + ) + else: + elapsed, response = time_call(partial(evaluate, backend_name)) + if reference_response is None: + reference_response = response + assert response == reference_response, "Native backend ciphertexts differ" + assert client.decode_hamming_client_packed(response, len(vectors)) == expected + timings[backend_name].append(elapsed) + order.append({"backend": backend_name, "repeat": repeat, "server_compute_s": elapsed}) + print( + f"{backend_name}: {elapsed:.6f} s; all distances verified" + + ("; identical native ciphertexts" if backend_name != "seal" else ""), + file=sys.stderr, + flush=True, + ) + + assert reference_response is not None + response_wire = packet(reference_response, len(vectors)) + # Profiles are separate extra runs: cProfile substantially distorts Python + # loop timings and must not be used to calculate benchmark speedups. + if args.profile_dir: + args.profile_dir.mkdir(parents=True, exist_ok=True) + for backend in backends: + profiler = cProfile.Profile() + response = profiler.runcall(evaluate, backend) + profiler.dump_stats(str(args.profile_dir / f"{backend}.prof")) + assert response == reference_response + + native_phases = {} + if args.native_profile: + from xtrace_sdk.x_vec.crypto.bfv_cpu_ext import _bfv_rns + + for backend in backends: + if variants[backend][0] in ("rns", "native", "residue"): + response, phases = _bfv_rns.profile_call(partial(evaluate, backend)) + assert response == reference_response + native_phases[backend] = phases + + summary = { + backend: { + "first_search_s": values[0], + "warm_searches_s": values[1:], + "warm_median_s": statistics.median(values[1:]) if len(values) > 1 else None, + } + for backend, values in timings.items() + } + source_paths = [ + Path(__file__).resolve(), + REPO_ROOT / "benchmarks/bfv_client_matrix.py", + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_client.py", + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py", + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py", + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py", + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py", + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.cpp"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.h"), + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile", + REPO_ROOT / "src/xtrace_sdk/x_vec/utils/xtrace_types.py", + ] + if any(b in ("rns", "native", "residue") for b, _ in variants.values()): + source_paths.extend((REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.so")) + if args.seal: + source_paths.append(REPO_ROOT / "experiments/bfv/packed_hamming.py") + output: dict[str, Any] = { + "environment": { + "utc": datetime.now(UTC).isoformat(), + "python": platform.python_version(), + "platform": platform.platform(), + "gmpy2": gmpy2.version(), + "gmp": gmpy2.mp_version(), + "revision": subprocess.run( + ["git", "rev-parse", "HEAD"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip(), + "working_tree_dirty": bool( + subprocess.run( + ["git", "status", "--porcelain"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + ), + }, + "command": sys.argv, + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, arithmetic context creation (reported under setup), key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "backend_variants": variants, + "config": config, + "ciphertext_modulus_hex": format(client._pk()["q"], "x"), + "ciphertext_modulus_primes": _rns_coefficient_primes( + args.poly_modulus_degree, args.coeff_modulus_bits + ) + if args.rns_modulus + else None, + "seed": args.seed, + "vectors": len(vectors), + "setup_timings": setup, + "runs": order, + "summary": summary, + "first_search_speedup": timings["reference"][0] / timings["optimized"][0] + if "reference" in timings and "optimized" in timings + else None, + "warm_median_speedup": statistics.median(timings["reference"][1:]) + / statistics.median(timings["optimized"][1:]) + if args.repeats > 1 and "reference" in timings and "optimized" in timings + else None, + "speedups_relative_to_first_native_backend": { + backend: { + "first": timings[backends[0]][0] / values[0], + "warm_median": statistics.median(timings[backends[0]][1:]) + / statistics.median(values[1:]) + if args.repeats > 1 + else None, + } + for backend, values in timings.items() + }, + "seal_comparison": seal_info, + "native_phases": native_phases, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": len(index_wire), + "public_keys_bytes": key_bytes, + "query_bytes": len(query_wire), + "response_bytes": len(response_wire), + "query_plus_response_bytes": len(query_wire) + len(response_wire), + }, + "caches": { + backend: { + **server._evaluator().cache_info(), + "server_plan_bytes": server._native().cache_bytes() + if variants[backend][0] in ("native", "residue") + else 0, + } + for backend, server in servers.items() + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": min( + BFV.noise_budget(BFV.ciphertext_from_ints(ct, client._pk()), client._keys()) + for ct in reference_response + ), + "identical_ciphertexts": True, + "all_distances_correct": True, + "top3": [ + {"index": i, "distance": expected[i]} + for i in sorted(range(len(vectors)), key=lambda i: (expected[i], i))[:3] + ], + "source_sha256": { + str(path.relative_to(REPO_ROOT)): hashlib.sha256(path.read_bytes()).hexdigest() + for path in source_paths + }, + } + print(json.dumps(output, indent=2)) + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text(json.dumps(output, indent=2) + "\n") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/bfv_verified.py b/benchmarks/bfv_verified.py new file mode 100644 index 0000000..6eb2cb5 --- /dev/null +++ b/benchmarks/bfv_verified.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +"""Measure authenticated/result-checked BFV sessions against the same raw inputs. + +Both servers use the unchanged residue kernels and identical keys/ciphertexts. +Run without concurrent benchmarks or tests. Only synthetic plaintext is seeded. +No key, ciphertext, verification seed or private fingerprint is written to JSON. +""" + +from __future__ import annotations + +import argparse +import hashlib +import importlib.metadata +import json +import platform +import secrets +import statistics +import subprocess +import sys +from datetime import UTC, datetime +from functools import partial +from pathlib import Path +from typing import Any + +import gmpy2 + +from bfv_client_matrix import REPO_ROOT, make_data, packet, time_call +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient +from xtrace_sdk.x_vec.crypto.bfv_security import ( + BFVExecutionPolicy, + _authenticated_body, + _decode_ciphertexts, + _encode_ciphertexts, + _unpack, +) +from xtrace_sdk.x_vec.crypto.bfv_verified_client import ( + BFVVerifiedClient, + BFVVerifiedServer, + _HammingSummary, +) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--num-vectors", type=int, default=1024) + parser.add_argument("--embed-len", type=int, default=512) + parser.add_argument("--seed", type=int, default=1337) + parser.add_argument("--repeats", type=int, default=6, help="Includes one cold search") + parser.add_argument("--json-out", type=Path) + args = parser.parse_args() + if args.num_vectors < 1 or args.embed_len < 1 or args.repeats < 2: + parser.error("positive dimensions and at least two repeats are required") + policy = BFVExecutionPolicy(embed_len=args.embed_len) + if args.num_vectors > policy.max_vectors or args.repeats > policy.max_queries: + parser.error("workload exceeds the local session policy") + vectors, query, expected = make_data(args.num_vectors, args.embed_len, args.seed) + timings: dict[str, list[float]] = { + name: [] for name in ("raw_server", "verified_server", "raw_client", "verified_client") + } + setup_times: dict[str, float] = {} + print("Preparing shared keys, encrypted index and private summary", file=sys.stderr, flush=True) + setup_times["key_generation_s"], private = time_call(lambda: BFVClient(**policy.config())) + auth = secrets.token_bytes(32) + client = BFVVerifiedClient(private, auth, policy=policy) + setup_times["encrypt_index_summary_export_auth_s"], setup = time_call( + lambda: client.prepare_index(vectors) + ) + setup_times["authenticate_import_setup_s"], verified = time_call( + lambda: BFVVerifiedServer(setup, auth, policy=policy) + ) + raw = BFVClient(**policy.config(), skip_key_gen=True, server_backend="residue") + # Trusted benchmark-only sharing of the same parsed PUBLIC key/index. Both + # server instances retain independent arithmetic contexts and lazy caches. + raw.public_key = verified._client._pk() + assert raw.keys is None and verified._client.keys is None + pk, index = raw._pk(), verified._index + setup_times["raw_arithmetic_context_s"], _ = time_call(raw._evaluator) + setup_times["verified_arithmetic_context_s"], _ = time_call(verified._client._evaluator) + setup_times["summary_only_preprocess_s"], _ = time_call( + lambda: _HammingSummary(vectors, args.embed_len, secrets.token_bytes(16)) + ) + summary_samples, runs, query_samples = [], [], [] + raw_response = b"" + response = b"" + request = b"" + query_wire = b"" + count = ( + len(vectors) + policy.params.poly_modulus_degree - 1 + ) // policy.params.poly_modulus_degree + + def raw_search(wire: bytes) -> bytes: + q = _decode_ciphertexts(wire, pk, 1, policy.max_query_bytes)[0] + result = raw.encode_hamming_server_packed(q, index, len(vectors)) + return _encode_ciphertexts(result, pk) + + def raw_finish(wire: bytes) -> list[int]: + cts = _decode_ciphertexts(wire, pk, count, policy.max_response_bytes) + return private.decode_hamming_client_packed(cts, len(vectors)) + + for repeat in range(args.repeats): + elapsed, request = time_call(lambda: client.begin_query(query)) + query_samples.append(elapsed) + query_wire = _unpack( + _authenticated_body(request, auth, 2, policy.max_query_bytes), + limit=policy.max_query_bytes, + )[2] + + order = ("raw", "verified") if repeat % 2 == 0 else ("verified", "raw") + for variant in order: + elapsed, result = time_call( + partial(raw_search, query_wire) + if variant == "raw" + else partial(verified.search, request) + ) + timings[f"{variant}_server"].append(elapsed) + runs.append({"repeat": repeat, "phase": f"{variant}_server", "seconds": elapsed}) + if variant == "raw": + raw_response = result + else: + response = result + print(f"{variant} server: {elapsed:.6f} s", file=sys.stderr, flush=True) + response_wire = _unpack( + _authenticated_body(response, auth, 3, policy.max_response_bytes), + limit=policy.max_response_bytes, + )[3] + assert response_wire == raw_response, "Wrapper changed the evaluated ciphertexts" + for variant in reversed(order): + elapsed, distances = time_call( + partial(raw_finish, raw_response) + if variant == "raw" + else partial(client.finish_query, response) + ) + assert distances == expected, "Incorrect Hamming distances" + timings[f"{variant}_client"].append(elapsed) + runs.append({"repeat": repeat, "phase": f"{variant}_client", "seconds": elapsed}) + assert client._summary is not None + elapsed, valid = time_call( + lambda: ( + client._summary is not None + and client._summary.matches(expected, client._summary.expected(query)) + ) + ) + assert valid + summary_samples.append(elapsed) + + summary = { + name: { + "first_s": samples[0], + "warm_samples_s": samples[1:], + "warm_median_s": statistics.median(samples[1:]), + } + for name, samples in timings.items() + } + setup_fields = _unpack( + _authenticated_body(setup, auth, 1, policy.max_setup_bytes), + limit=policy.max_setup_bytes, + array_limit=policy.max_vectors, + ) + public_bytes, index_bytes = len(setup_fields[2]), len(setup_fields[5]) + query_cts = _decode_ciphertexts(query_wire, pk, 1, policy.max_query_bytes) + response_cts = _decode_ciphertexts(raw_response, pk, count, policy.max_response_bytes) + protected_s, protected = time_call(lambda: client.protect_state(secrets.token_bytes(32))) + sources = [ + Path(__file__).resolve(), + REPO_ROOT / "benchmarks/bfv_client_matrix.py", + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto").glob("bfv*.py"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/encryption").glob("bfv*.py"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.cpp"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.h"), + *(REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext").glob("*.so"), + REPO_ROOT / "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile", + REPO_ROOT / "src/xtrace_sdk/x_vec/utils/xtrace_types.py", + ] + output: dict[str, Any] = { + "environment": { + "utc": datetime.now(UTC).isoformat(), + "python": platform.python_version(), + "platform": platform.platform(), + "gmpy2": gmpy2.version(), + "gmp": gmpy2.mp_version(), + "pycryptodome": importlib.metadata.version("pycryptodome"), + "msgpack": importlib.metadata.version("msgpack"), + "revision": subprocess.run( + ["git", "rev-parse", "HEAD"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip(), + "working_tree_dirty": bool( + subprocess.run( + ["git", "status", "--porcelain"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + ), + }, + "command": sys.argv, + "config": policy.config(), + "seed": args.seed, + "vectors": len(vectors), + "measurement_notes": "Identical keys and ciphertexts, independent public-only residue server contexts. Both server timers include MessagePack ciphertext decode/encode and native arithmetic; verified additionally includes authentication, session checks and outer framing. Index import, arithmetic context creation, encryption and private decryption are excluded from server times. First search has cold key/mask caches; five warm samples by default, alternating order. Client timers include ciphertext decode/decrypt/slot decode; verified adds authentication, ticket handling and private result checking. Query encryption is separately timed outside these phases. No network, remote endpoint, SEAL, security-level claim or constant-time claim.", + "setup_timings": setup_times, + "runs": runs, + "summary": summary, + "query_encrypt_and_bind_samples_s": query_samples, + "summary_only_check_samples_s": summary_samples, + "protect_private_state_s": protected_s, + "sizes": { + "public_keys_bytes": public_bytes, + "index_ciphertext_array_bytes": index_bytes, + "authenticated_setup_bytes": len(setup), + "raw_query_array_bytes": len(query_wire), + "raw_response_array_bytes": len(raw_response), + "authenticated_query_bytes": len(request), + "authenticated_response_bytes": len(response), + "authenticated_query_plus_response_bytes": len(request) + len(response), + "query_authentication_and_framing_overhead_bytes": len(request) - len(query_wire), + "response_authentication_and_framing_overhead_bytes": len(response) - len(raw_response), + "private_verification_payload_bytes": client.verification_state_bytes, + "protected_private_state_bytes": len(protected), + "previous_benchmark_encrypted_index_bytes": len(packet(index, len(vectors))), + "previous_benchmark_query_bytes": len(packet(query_cts, 1)), + "previous_benchmark_response_bytes": len(packet(response_cts, len(vectors))), + }, + "identical_ciphertexts": True, + "all_distances_correct": True, + "source_sha256": { + str(p.relative_to(REPO_ROOT)): hashlib.sha256(p.read_bytes()).hexdigest() + for p in sources + }, + } + encoded = json.dumps(output, indent=2) + print(encoded) + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text(encoded + "\n") + + +if __name__ == "__main__": + main() diff --git a/benchmarks/results/.gitignore b/benchmarks/results/.gitignore new file mode 100644 index 0000000..0c7792e --- /dev/null +++ b/benchmarks/results/.gitignore @@ -0,0 +1,6 @@ +# The Nitro protocol evidence is intended to accompany its implementation. +!native_bfv_nitro_local_1024.json +!native_bfv_nitro_audit_1024.json +# Paired 8,192-vector I/O measurements, with both BFV parameter profiles. +!native_bfv_8192.json +!native_bfv_review_8192.json diff --git a/benchmarks/results/native_bfv_1024.json b/benchmarks/results/native_bfv_1024.json new file mode 100644 index 0000000..d8e98eb --- /dev/null +++ b/benchmarks/results/native_bfv_1024.json @@ -0,0 +1,177 @@ +{ + "environment": { + "utc": "2026-09-10T20:27:09.027921+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "04204ec9b98923d90ad4e6f489bac25c3f46aa16", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_client_matrix.py", + "--num-vectors", + "1024", + "--json-out", + "benchmarks/results/native_bfv_1024.json" + ], + "measurement_notes": "CPU synthetic experiment; no equivalent-security claim. Keys are public JSON including BFV evaluation keys. Index/query/response use common MessagePack framing with LE integer bytes and implicit IDs. No network/HTTP/TLS, secret-key, plaintext-content or AES-content bytes. Timings are separate from serialization; no concurrent test/benchmark processes intended.", + "seed": 1337, + "runs": [ + { + "variant": "bfv-packed", + "vectors": 1024, + "embed_len": 512, + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "compact": true, + "timings": { + "key_generation_s": 4.32266186689958, + "public_key_export_s": 0.7880964139476418, + "public_key_import_s": 0.853526117047295, + "index_encrypt_s": 6.748511495068669, + "query_encrypt_s": 0.1036944561637938, + "index_serialize_s": 0.038064548978582025, + "query_serialize_s": 0.0002170950174331665, + "index_deserialize_s": 0.022240569116547704, + "query_deserialize_s": 0.00021161511540412903, + "server_compute_s": 108.29552126303315, + "response_serialize_s": 0.0003346470184624195, + "response_deserialize_s": 6.959214806556702e-05, + "client_decode_s": 0.0412755839060992 + }, + "sizes": { + "encrypted_index_bytes": 23598496, + "public_keys_bytes": 85602812, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "index_ciphertexts": 64, + "response_ciphertexts": 1, + "minimum_response_noise_budget_bits": 26, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "correct": true, + "repeat": 0 + }, + { + "variant": "paillier-cpu", + "vectors": 1024, + "embed_len": 512, + "config": { + "embed_len": 512, + "key_len": 1024 + }, + "compact": false, + "timings": { + "key_generation_s": 0.5261988120619208, + "public_key_export_s": 4.5150984078645706e-05, + "public_key_import_s": 1.7459969967603683e-05, + "index_encrypt_s": 10.52772010397166, + "query_encrypt_s": 0.010291318176314235, + "index_serialize_s": 0.001604192890226841, + "query_serialize_s": 1.933099702000618e-05, + "index_deserialize_s": 0.0005795308388769627, + "query_deserialize_s": 1.3759825378656387e-05, + "server_compute_s": 0.005445469869300723, + "response_serialize_s": 0.0007037140894681215, + "response_deserialize_s": 0.0005315609741955996, + "client_decode_s": 7.0446601149160415 + }, + "sizes": { + "encrypted_index_bytes": 528396, + "public_keys_bytes": 2502, + "query_bytes": 544, + "response_bytes": 528403, + "query_plus_response_bytes": 528947 + }, + "index_ciphertexts": 1024, + "response_ciphertexts": 1024, + "minimum_response_noise_budget_bits": null, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "correct": true, + "repeat": 0 + } + ], + "source_sha256": { + "benchmarks/bfv_client_matrix.py": "c5d1f2eafe6941219c27c2501f2f753579bcbe8cce871a9f23a9591367424658", + "src/xtrace_sdk/x_vec/crypto/paillier_client.py": "40582d7acb86ee7dd2060e2d795c76829f4879db3b1eca9518be7a9599de4788", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "1f55ce8061c18644f98e99d683351bc849ef78fb917ac511c9a08eddff214d73", + "src/xtrace_sdk/x_vec/crypto/goldwasser_micali_client.py": "0982ad7532d07c8e26a701359ce8f4f7d4c1a3beb714050e741f3b534d59935f", + "src/xtrace_sdk/x_vec/crypto/paillier_lookup_client.py": "4faa0dd71d03b3e9bc9e49c4e60e8be7b173ff514b52cf1d94a3b1a2426bf699", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier_lookup.py": "eed0be7e0e2c1f6e8e8c0138262ec071fef73648e334a948ef1f82df6e9b9712", + "src/xtrace_sdk/x_vec/crypto/encryption/__init__.py": "261a50ed1758cee38c880cc625187f8acb051c6d5c96ba7a71ee956bdbea2cc5", + "src/xtrace_sdk/x_vec/crypto/encryption/goldwasser_micali.py": "4b32296c6210663c098718fa49bd810c427b1f8102d026d12a7d734f58e8e3d1", + "src/xtrace_sdk/x_vec/crypto/encryption/homomorphic_base.py": "915da9e500d0a142651243bb13de7631dc96103fea2ec9e5b9288bee498010e7", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier.py": "525a1944caef38d5d216ae84fcac982627d2a2fddea50b747e47ef05e2e91c9f", + "src/xtrace_sdk/x_vec/crypto/encryption/aes.py": "2c833351db3b249c381e315307d858791ee4e5b633d3465127cad763b4d095d5", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3" + }, + "median_timings": { + "bfv-packed": { + "key_generation_s": 4.32266186689958, + "public_key_export_s": 0.7880964139476418, + "public_key_import_s": 0.853526117047295, + "index_encrypt_s": 6.748511495068669, + "query_encrypt_s": 0.1036944561637938, + "index_serialize_s": 0.038064548978582025, + "query_serialize_s": 0.0002170950174331665, + "index_deserialize_s": 0.022240569116547704, + "query_deserialize_s": 0.00021161511540412903, + "server_compute_s": 108.29552126303315, + "response_serialize_s": 0.0003346470184624195, + "response_deserialize_s": 6.959214806556702e-05, + "client_decode_s": 0.0412755839060992 + }, + "paillier-cpu": { + "key_generation_s": 0.5261988120619208, + "public_key_export_s": 4.5150984078645706e-05, + "public_key_import_s": 1.7459969967603683e-05, + "index_encrypt_s": 10.52772010397166, + "query_encrypt_s": 0.010291318176314235, + "index_serialize_s": 0.001604192890226841, + "query_serialize_s": 1.933099702000618e-05, + "index_deserialize_s": 0.0005795308388769627, + "query_deserialize_s": 1.3759825378656387e-05, + "server_compute_s": 0.005445469869300723, + "response_serialize_s": 0.0007037140894681215, + "response_deserialize_s": 0.0005315609741955996, + "client_decode_s": 7.0446601149160415 + } + } +} diff --git a/benchmarks/results/native_bfv_8192.json b/benchmarks/results/native_bfv_8192.json new file mode 100644 index 0000000..9500a67 --- /dev/null +++ b/benchmarks/results/native_bfv_8192.json @@ -0,0 +1,210 @@ +{ + "environment": { + "utc": "2026-09-17T19:10:37.064659+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "be71b0dd4fa3693f6a6628e61f31e4b54ece9760", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_client_matrix.py", + "--num-vectors", + "8192", + "--embed-len", + "512", + "--poly-modulus-degree", + "8192", + "--server-backend", + "residue", + "--rns-modulus", + "--variants", + "bfv-packed,paillier-cpu", + "--repeats", + "1", + "--json-out", + "benchmarks/results/native_bfv_8192.json" + ], + "measurement_notes": "CPU synthetic experiment; no equivalent-security claim. Keys are public JSON including BFV evaluation keys. Index/query/response use common MessagePack framing with LE integer bytes and implicit IDs. No network/HTTP/TLS, secret-key, plaintext-content or AES-content bytes. Timings are separate from serialization; no concurrent test/benchmark processes intended.", + "seed": 1337, + "runs": [ + { + "variant": "bfv-packed", + "server_backend": "residue", + "vectors": 8192, + "embed_len": 512, + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "compact": true, + "timings": { + "key_generation_s": 4.473671418149024, + "public_key_export_s": 0.7987699322402477, + "public_key_import_s": 1.4380085780285299, + "index_encrypt_s": 54.573941993992776, + "query_encrypt_s": 0.10487454803660512, + "index_serialize_s": 0.38077715784311295, + "query_serialize_s": 0.0003552851267158985, + "index_deserialize_s": 0.2512285150587559, + "query_deserialize_s": 0.0002061929553747177, + "server_compute_s": 19.836358953267336, + "response_serialize_s": 8.567096665501595e-05, + "response_deserialize_s": 7.523223757743835e-05, + "client_decode_s": 0.04191199177876115 + }, + "sizes": { + "encrypted_index_bytes": 188787740, + "public_keys_bytes": 85603562, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "index_ciphertexts": 512, + "response_ciphertexts": 1, + "minimum_response_noise_budget_bits": 26, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 8191, + "distance": 0 + }, + { + "index": 2654, + "distance": 217 + } + ], + "correct": true, + "repeat": 0 + }, + { + "variant": "paillier-cpu", + "server_backend": null, + "vectors": 8192, + "embed_len": 512, + "config": { + "embed_len": 512, + "key_len": 1024 + }, + "compact": false, + "timings": { + "key_generation_s": 0.22122686309739947, + "public_key_export_s": 4.7971028834581375e-05, + "public_key_import_s": 2.322997897863388e-05, + "index_encrypt_s": 86.23786324588582, + "query_encrypt_s": 0.010293608997017145, + "index_serialize_s": 0.012443804182112217, + "query_serialize_s": 3.6510173231363297e-05, + "index_deserialize_s": 0.004360671620815992, + "query_deserialize_s": 2.3310072720050812e-05, + "server_compute_s": 0.043896660674363375, + "response_serialize_s": 0.005283337086439133, + "response_deserialize_s": 0.00477636931464076, + "client_decode_s": 57.76321804802865 + }, + "sizes": { + "encrypted_index_bytes": 4226979, + "public_keys_bytes": 2502, + "query_bytes": 544, + "response_bytes": 4226979, + "query_plus_response_bytes": 4227523 + }, + "index_ciphertexts": 8192, + "response_ciphertexts": 8192, + "minimum_response_noise_budget_bits": null, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 8191, + "distance": 0 + }, + { + "index": 2654, + "distance": 217 + } + ], + "correct": true, + "repeat": 0 + } + ], + "source_sha256": { + "benchmarks/bfv_client_matrix.py": "2f93923f21f3b9165ddc510ce029d61e8df873c7b519c1d61a99a193697a2f38", + "src/xtrace_sdk/x_vec/crypto/bfv_attested_client.py": "d3bed7a596fd19f834ea308fc332f0a28109a1136c7dd63026819b4cc106c7de", + "src/xtrace_sdk/x_vec/crypto/paillier_client.py": "40582d7acb86ee7dd2060e2d795c76829f4879db3b1eca9518be7a9599de4788", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0", + "src/xtrace_sdk/x_vec/crypto/goldwasser_micali_client.py": "0982ad7532d07c8e26a701359ce8f4f7d4c1a3beb714050e741f3b534d59935f", + "src/xtrace_sdk/x_vec/crypto/paillier_lookup_client.py": "4faa0dd71d03b3e9bc9e49c4e60e8be7b173ff514b52cf1d94a3b1a2426bf699", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier_lookup.py": "eed0be7e0e2c1f6e8e8c0138262ec071fef73648e334a948ef1f82df6e9b9712", + "src/xtrace_sdk/x_vec/crypto/encryption/__init__.py": "261a50ed1758cee38c880cc625187f8acb051c6d5c96ba7a71ee956bdbea2cc5", + "src/xtrace_sdk/x_vec/crypto/encryption/goldwasser_micali.py": "4b32296c6210663c098718fa49bd810c427b1f8102d026d12a7d734f58e8e3d1", + "src/xtrace_sdk/x_vec/crypto/encryption/homomorphic_base.py": "915da9e500d0a142651243bb13de7631dc96103fea2ec9e5b9288bee498010e7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier.py": "525a1944caef38d5d216ae84fcac982627d2a2fddea50b747e47ef05e2e91c9f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_private.py": "b1291e6d017be84638b054fb84e651734a44b2d54d601eb50dae2e9d2016bdfa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/encryption/aes.py": "2c833351db3b249c381e315307d858791ee4e5b633d3465127cad763b4d095d5", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_decoder.h": "66aceec7cc8452dabdde0061d347195e49a9500fba7bcbbe805c967b7171d767", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp": "7b48d92e526147d97d9401e868c8d3e9dbf0b8ba567a67e05047e01f2a371fc0", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ee3f7371f92ba9fa3ebb8037906fc23c5f7832f96908d5879f1a4326847ffb53", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + }, + "median_timings": { + "bfv-packed": { + "key_generation_s": 4.473671418149024, + "public_key_export_s": 0.7987699322402477, + "public_key_import_s": 1.4380085780285299, + "index_encrypt_s": 54.573941993992776, + "query_encrypt_s": 0.10487454803660512, + "index_serialize_s": 0.38077715784311295, + "query_serialize_s": 0.0003552851267158985, + "index_deserialize_s": 0.2512285150587559, + "query_deserialize_s": 0.0002061929553747177, + "server_compute_s": 19.836358953267336, + "response_serialize_s": 8.567096665501595e-05, + "response_deserialize_s": 7.523223757743835e-05, + "client_decode_s": 0.04191199177876115 + }, + "paillier-cpu": { + "key_generation_s": 0.22122686309739947, + "public_key_export_s": 4.7971028834581375e-05, + "public_key_import_s": 2.322997897863388e-05, + "index_encrypt_s": 86.23786324588582, + "query_encrypt_s": 0.010293608997017145, + "index_serialize_s": 0.012443804182112217, + "query_serialize_s": 3.6510173231363297e-05, + "index_deserialize_s": 0.004360671620815992, + "query_deserialize_s": 2.3310072720050812e-05, + "server_compute_s": 0.043896660674363375, + "response_serialize_s": 0.005283337086439133, + "response_deserialize_s": 0.00477636931464076, + "client_decode_s": 57.76321804802865 + } + } +} diff --git a/benchmarks/results/native_bfv_cpp_server_1024.json b/benchmarks/results/native_bfv_cpp_server_1024.json new file mode 100644 index 0000000..fe9d1b3 --- /dev/null +++ b/benchmarks/results/native_bfv_cpp_server_1024.json @@ -0,0 +1,388 @@ +{ + "environment": { + "utc": "2026-09-11T01:33:24.194247+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "79ad443f2cc4606af8f142af075ecdcf8e62d8a2", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--backends", + "rns,native", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_cpp_server_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.330939290113747, + "index_encrypt_s": 6.664838324999437, + "query_encrypt_s": 0.10223700385540724, + "public_key_export_s": 0.8034978481009603, + "rns_public_key_import_s": 0.8361312141641974, + "native_public_key_import_s": 0.8487555379979312 + }, + "runs": [ + { + "backend": "rns", + "repeat": 0, + "server_compute_s": 8.21241048607044 + }, + { + "backend": "native", + "repeat": 0, + "server_compute_s": 5.681734326994047 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 2.366231924155727 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 2.358917977893725 + }, + { + "backend": "native", + "repeat": 1, + "server_compute_s": 5.342603240860626 + }, + { + "backend": "rns", + "repeat": 1, + "server_compute_s": 7.938400606857613 + }, + { + "backend": "rns", + "repeat": 2, + "server_compute_s": 7.9265675991773605 + }, + { + "backend": "native", + "repeat": 2, + "server_compute_s": 5.330243221018463 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 2.363118221051991 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 2.359207842964679 + }, + { + "backend": "native", + "repeat": 3, + "server_compute_s": 5.342728961026296 + }, + { + "backend": "rns", + "repeat": 3, + "server_compute_s": 7.920907473890111 + } + ], + "summary": { + "rns": { + "first_search_s": 8.21241048607044, + "warm_searches_s": [ + 7.938400606857613, + 7.9265675991773605, + 7.920907473890111 + ], + "warm_median_s": 7.9265675991773605 + }, + "native": { + "first_search_s": 5.681734326994047, + "warm_searches_s": [ + 5.342603240860626, + 5.330243221018463, + 5.342728961026296 + ], + "warm_median_s": 5.342603240860626 + }, + "seal": { + "first_search_s": 2.366231924155727, + "warm_searches_s": [ + 2.358917977893725, + 2.363118221051991, + 2.359207842964679 + ], + "warm_median_s": 2.359207842964679 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "rns": { + "first": 1.0, + "warm_median": 1.0 + }, + "native": { + "first": 1.4454055774929662, + "warm_median": 1.4836526767614677 + }, + "seal": { + "first": 3.4706701410938963, + "warm_median": 3.3598428484437832 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 27678953, + "query_bytes": 432588, + "public_key_bytes": 39586623, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 105880, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "rns": { + "forward_ntt": { + "seconds": 1.561838909, + "calls": 18792 + }, + "inverse_ntt": { + "seconds": 0.759356965, + "calls": 7480 + }, + "to_residues": { + "seconds": 0.144149786, + "calls": 384 + }, + "crt_reconstruct": { + "seconds": 1.3842246550000001, + "calls": 1726 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.05560698100000001, + "calls": 703 + }, + "pointwise": { + "seconds": 0.421228254, + "calls": 895 + }, + "scale_round": { + "seconds": 0.159821076, + "calls": 192 + }, + "mod_q": { + "seconds": 0.485560737, + "calls": 1534 + }, + "buffers": { + "seconds": 0.026597349000000003, + "calls": 1151 + }, + "automorphism": { + "seconds": 0.35820702000000004, + "calls": 639 + }, + "add_sub": { + "seconds": 0.19624586500000002, + "calls": 704 + }, + "wire_import": { + "seconds": 0.0, + "calls": 0 + }, + "wire_export": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 2.388239892, + "calls": 1 + } + }, + "native": { + "forward_ntt": { + "seconds": 1.564787522, + "calls": 18280 + }, + "inverse_ntt": { + "seconds": 0.5568240910000001, + "calls": 7480 + }, + "to_residues": { + "seconds": 0.12858792700000002, + "calls": 256 + }, + "crt_reconstruct": { + "seconds": 0.865034472, + "calls": 1726 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.053277151, + "calls": 703 + }, + "pointwise": { + "seconds": 0.407613388, + "calls": 895 + }, + "scale_round": { + "seconds": 0.16121975600000002, + "calls": 193 + }, + "mod_q": { + "seconds": 0.49114541400000006, + "calls": 1534 + }, + "buffers": { + "seconds": 0.024296814, + "calls": 1023 + }, + "automorphism": { + "seconds": 0.35786369900000004, + "calls": 639 + }, + "add_sub": { + "seconds": 0.211706636, + "calls": 767 + }, + "wire_import": { + "seconds": 0.028895339000000003, + "calls": 130 + }, + "wire_export": { + "seconds": 0.00016453300000000002, + "calls": 2 + }, + "other": { + "seconds": 0.502915741, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 23598495, + "public_keys_bytes": 85603219, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "rns": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": false, + "switch_keys": 16, + "key_payload_bytes": 50331648, + "server_plan_bytes": 0 + }, + "native": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "switch_keys": 18, + "key_payload_bytes": 56623104, + "server_plan_bytes": 655360 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "6e8187062aef810a0cc25c4e3689a99ba79ee29ea5048fe6aa297daf6273dd40", + "benchmarks/bfv_client_matrix.py": "925f50c0e21c591d293c308e74dd5b5ab9ee9c6016543c956dbfce8b6ecec502", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "67ee7ebcce11b4b4bf55297a32811904a285c7c3f4edd884e61ef85af88f2636", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "88c944adbf6a717cbf32e0a62d7e7d9f648f5db0d92e954c8a2e1307eeb45d81", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "96ce615ee4b187c45d3118c1a17df679d5ad9cbe2f233783e2a15ed43bd1aeff", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "176a4d01c38e0e20aa9e6ed2dd78bc5a88f50d5262b5a9434307d254befeab50", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "3931b73d1622443476cef5f9de2af03819364dd3cd54c5efe5e2ed49acd136f7", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "e019dad8d819626f051a31519557b26ff65648d0327e28ddcd943d2344f57b8a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "88ec7b5295f2916f45a531d695e6c18f91b3b8dddcae0ada8c0d4527a56ed4f8", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "26221a0fd074abd1251c03e5b54a7db3565e024ec942be94da10d6a1606e0233", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ba256b062d975310c3de166cd5242c1cea5254250a769b942b0a49f5a7aed434", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "c86ff6e20fe63db5599baaa086f87dbb3f33d956fa175388e054b79d765c8e0c", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_cpp_server_32.json b/benchmarks/results/native_bfv_cpp_server_32.json new file mode 100644 index 0000000..d2847f5 --- /dev/null +++ b/benchmarks/results/native_bfv_cpp_server_32.json @@ -0,0 +1,388 @@ +{ + "environment": { + "utc": "2026-09-11T01:31:51.513445+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "79ad443f2cc4606af8f142af075ecdcf8e62d8a2", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "32", + "--backends", + "rns,native", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_cpp_server_32.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 32, + "setup_timings": { + "key_generation_s": 4.342289729043841, + "index_encrypt_s": 0.21233072807081044, + "query_encrypt_s": 0.10064343316480517, + "public_key_export_s": 0.7829922570381314, + "rns_public_key_import_s": 0.856646987143904, + "native_public_key_import_s": 0.851454121991992 + }, + "runs": [ + { + "backend": "rns", + "repeat": 0, + "server_compute_s": 0.4936013799160719 + }, + { + "backend": "native", + "repeat": 0, + "server_compute_s": 0.5134266279637814 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 0.0763197538908571 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 0.07405148795805871 + }, + { + "backend": "native", + "repeat": 1, + "server_compute_s": 0.16401935601606965 + }, + { + "backend": "rns", + "repeat": 1, + "server_compute_s": 0.26051225513219833 + }, + { + "backend": "rns", + "repeat": 2, + "server_compute_s": 0.2602035889867693 + }, + { + "backend": "native", + "repeat": 2, + "server_compute_s": 0.1639279848895967 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 0.0742087010294199 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 0.07412873883731663 + }, + { + "backend": "native", + "repeat": 3, + "server_compute_s": 0.1639478448778391 + }, + { + "backend": "rns", + "repeat": 3, + "server_compute_s": 0.2600402070675045 + } + ], + "summary": { + "rns": { + "first_search_s": 0.4936013799160719, + "warm_searches_s": [ + 0.26051225513219833, + 0.2602035889867693, + 0.2600402070675045 + ], + "warm_median_s": 0.2602035889867693 + }, + "native": { + "first_search_s": 0.5134266279637814, + "warm_searches_s": [ + 0.16401935601606965, + 0.1639279848895967, + 0.1639478448778391 + ], + "warm_median_s": 0.1639478448778391 + }, + "seal": { + "first_search_s": 0.0763197538908571, + "warm_searches_s": [ + 0.07405148795805871, + 0.0742087010294199, + 0.07412873883731663 + ], + "warm_median_s": 0.07412873883731663 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "rns": { + "first": 1.0, + "warm_median": 1.0 + }, + "native": { + "first": 0.9613864046624633, + "warm_median": 1.5871119817442698 + }, + "seal": { + "first": 6.467544177644472, + "warm_median": 3.5101580448820755 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 865239, + "query_bytes": 432560, + "public_key_bytes": 39586592, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 103312, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "rns": { + "forward_ntt": { + "seconds": 0.046839558, + "calls": 564 + }, + "inverse_ntt": { + "seconds": 0.022890933000000002, + "calls": 226 + }, + "to_residues": { + "seconds": 0.004476374, + "calls": 12 + }, + "crt_reconstruct": { + "seconds": 0.041671711, + "calls": 52 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.0015517270000000001, + "calls": 21 + }, + "pointwise": { + "seconds": 0.012541273, + "calls": 27 + }, + "scale_round": { + "seconds": 0.004987500000000001, + "calls": 6 + }, + "mod_q": { + "seconds": 0.014520551000000001, + "calls": 46 + }, + "buffers": { + "seconds": 0.000788527, + "calls": 35 + }, + "automorphism": { + "seconds": 0.010529403000000001, + "calls": 19 + }, + "add_sub": { + "seconds": 0.006080952000000001, + "calls": 22 + }, + "wire_import": { + "seconds": 0.0, + "calls": 0 + }, + "wire_export": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.09332303, + "calls": 1 + } + }, + "native": { + "forward_ntt": { + "seconds": 0.047080215, + "calls": 548 + }, + "inverse_ntt": { + "seconds": 0.016968431000000003, + "calls": 226 + }, + "to_residues": { + "seconds": 0.004014191, + "calls": 8 + }, + "crt_reconstruct": { + "seconds": 0.026213566, + "calls": 52 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.0016716600000000002, + "calls": 21 + }, + "pointwise": { + "seconds": 0.012280482, + "calls": 27 + }, + "scale_round": { + "seconds": 0.006211254, + "calls": 7 + }, + "mod_q": { + "seconds": 0.014689607, + "calls": 46 + }, + "buffers": { + "seconds": 0.00095556, + "calls": 31 + }, + "automorphism": { + "seconds": 0.010587844, + "calls": 19 + }, + "add_sub": { + "seconds": 0.006361085000000001, + "calls": 23 + }, + "wire_import": { + "seconds": 0.0013337070000000002, + "calls": 6 + }, + "wire_export": { + "seconds": 0.000158364, + "calls": 2 + }, + "other": { + "seconds": 0.015787336000000003, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 737480, + "public_keys_bytes": 85603223, + "query_bytes": 368754, + "response_bytes": 102494, + "query_plus_response_bytes": 471248 + }, + "caches": { + "rns": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": false, + "switch_keys": 11, + "key_payload_bytes": 34603008, + "server_plan_bytes": 0 + }, + "native": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "switch_keys": 18, + "key_payload_bytes": 56623104, + "server_plan_bytes": 655360 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 31, + "distance": 0 + }, + { + "index": 19, + "distance": 241 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "6e8187062aef810a0cc25c4e3689a99ba79ee29ea5048fe6aa297daf6273dd40", + "benchmarks/bfv_client_matrix.py": "925f50c0e21c591d293c308e74dd5b5ab9ee9c6016543c956dbfce8b6ecec502", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "67ee7ebcce11b4b4bf55297a32811904a285c7c3f4edd884e61ef85af88f2636", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "88c944adbf6a717cbf32e0a62d7e7d9f648f5db0d92e954c8a2e1307eeb45d81", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "96ce615ee4b187c45d3118c1a17df679d5ad9cbe2f233783e2a15ed43bd1aeff", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "176a4d01c38e0e20aa9e6ed2dd78bc5a88f50d5262b5a9434307d254befeab50", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "3931b73d1622443476cef5f9de2af03819364dd3cd54c5efe5e2ed49acd136f7", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "e019dad8d819626f051a31519557b26ff65648d0327e28ddcd943d2344f57b8a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "88ec7b5295f2916f45a531d695e6c18f91b3b8dddcae0ada8c0d4527a56ed4f8", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "26221a0fd074abd1251c03e5b54a7db3565e024ec942be94da10d6a1606e0233", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ba256b062d975310c3de166cd5242c1cea5254250a769b942b0a49f5a7aed434", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "c86ff6e20fe63db5599baaa086f87dbb3f33d956fa175388e054b79d765c8e0c", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_fused_1024.json b/benchmarks/results/native_bfv_fused_1024.json new file mode 100644 index 0000000..c2d35ae --- /dev/null +++ b/benchmarks/results/native_bfv_fused_1024.json @@ -0,0 +1,562 @@ +{ + "environment": { + "utc": "2026-09-11T03:04:45.774330+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "e5c075106c91d52d927af65e3d324f516c8c53b0", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--rns-modulus", + "--backends", + "residue", + "--residue-levels", + "0,1,2", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_fused_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, arithmetic context creation (reported under setup), key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "backend_variants": { + "residue-0": [ + "residue", + 0 + ], + "residue-1": [ + "residue", + 1 + ], + "residue-2": [ + "residue", + 2 + ] + }, + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "ffffffffffbc0030000004bff78002fff1004bffbc001", + "ciphertext_modulus_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.47719966317527, + "index_encrypt_s": 6.908126533031464, + "query_encrypt_s": 0.10573444119654596, + "public_key_export_s": 0.8328375099226832, + "residue-0_public_key_import_s": 0.874006818048656, + "residue-0_arithmetic_context_s": 0.001661886926740408, + "residue-1_public_key_import_s": 0.8809227580204606, + "residue-1_arithmetic_context_s": 0.0008880100212991238, + "residue-2_public_key_import_s": 0.8828790578991175, + "residue-2_arithmetic_context_s": 0.0008801498916000128 + }, + "runs": [ + { + "backend": "residue-0", + "repeat": 0, + "server_compute_s": 3.12221663701348 + }, + { + "backend": "residue-1", + "repeat": 0, + "server_compute_s": 2.9633277570828795 + }, + { + "backend": "residue-2", + "repeat": 0, + "server_compute_s": 2.7569567230530083 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 2.458954274887219 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 2.4774899801705033 + }, + { + "backend": "residue-2", + "repeat": 1, + "server_compute_s": 2.4064054479822516 + }, + { + "backend": "residue-1", + "repeat": 1, + "server_compute_s": 2.645552079193294 + }, + { + "backend": "residue-0", + "repeat": 1, + "server_compute_s": 2.7930076429620385 + }, + { + "backend": "residue-0", + "repeat": 2, + "server_compute_s": 2.799179068999365 + }, + { + "backend": "residue-1", + "repeat": 2, + "server_compute_s": 2.645475624827668 + }, + { + "backend": "residue-2", + "repeat": 2, + "server_compute_s": 2.4265225289855152 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 2.4921487709507346 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 2.4789128969423473 + }, + { + "backend": "residue-2", + "repeat": 3, + "server_compute_s": 2.4004873901139945 + }, + { + "backend": "residue-1", + "repeat": 3, + "server_compute_s": 2.638776435982436 + }, + { + "backend": "residue-0", + "repeat": 3, + "server_compute_s": 2.7942729440983385 + } + ], + "summary": { + "residue-0": { + "first_search_s": 3.12221663701348, + "warm_searches_s": [ + 2.7930076429620385, + 2.799179068999365, + 2.7942729440983385 + ], + "warm_median_s": 2.7942729440983385 + }, + "residue-1": { + "first_search_s": 2.9633277570828795, + "warm_searches_s": [ + 2.645552079193294, + 2.645475624827668, + 2.638776435982436 + ], + "warm_median_s": 2.645475624827668 + }, + "residue-2": { + "first_search_s": 2.7569567230530083, + "warm_searches_s": [ + 2.4064054479822516, + 2.4265225289855152, + 2.4004873901139945 + ], + "warm_median_s": 2.4064054479822516 + }, + "seal": { + "first_search_s": 2.458954274887219, + "warm_searches_s": [ + 2.4774899801705033, + 2.4921487709507346, + 2.4789128969423473 + ], + "warm_median_s": 2.4789128969423473 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "residue-0": { + "first": 1.0, + "warm_median": 1.0 + }, + "residue-1": { + "first": 1.0536183955861202, + "warm_median": 1.0562459611701633 + }, + "residue-2": { + "first": 1.1324866331438053, + "warm_median": 1.1611812741037926 + }, + "seal": { + "first": 1.269733507816725, + "warm_median": 1.1272170746882542 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 27681091, + "query_bytes": 432663, + "public_key_bytes": 39585150, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 105961, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "residue-0": { + "forward_ntt": { + "seconds": 1.195090398, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.46048345, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.138519197, + "calls": 320 + }, + "crt_reconstruct": { + "seconds": 0.18281814100000002, + "calls": 194 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.07851556500000001, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.29695649500000004, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.16776511300000002, + "calls": 193 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.017771753, + "calls": 895 + }, + "automorphism": { + "seconds": 0.031365696000000005, + "calls": 639 + }, + "add_sub": { + "seconds": 0.058091784, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.029809831000000002, + "calls": 130 + }, + "wire_export": { + "seconds": 0.00016202300000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.060537628, + "calls": 705 + }, + "rns_scale": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.08418196400000001, + "calls": 1 + } + }, + "residue-1": { + "forward_ntt": { + "seconds": 1.116043798, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.399708159, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.141233763, + "calls": 320 + }, + "crt_reconstruct": { + "seconds": 0.188225854, + "calls": 194 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.07928863300000001, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.298050792, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.173725757, + "calls": 193 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.01836928, + "calls": 895 + }, + "automorphism": { + "seconds": 0.032022763, + "calls": 639 + }, + "add_sub": { + "seconds": 0.059361043, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.030430777000000003, + "calls": 130 + }, + "wire_export": { + "seconds": 0.000163553, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.060993114, + "calls": 705 + }, + "rns_scale": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.085690057, + "calls": 1 + } + }, + "residue-2": { + "forward_ntt": { + "seconds": 1.108225623, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.396151433, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.08412324700000001, + "calls": 128 + }, + "crt_reconstruct": { + "seconds": 0.00049237, + "calls": 2 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.077781504, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.294543106, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.001360969, + "calls": 1 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.020847512000000002, + "calls": 895 + }, + "automorphism": { + "seconds": 0.03390283, + "calls": 639 + }, + "add_sub": { + "seconds": 0.058100396000000006, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.029341752000000002, + "calls": 130 + }, + "wire_export": { + "seconds": 0.000183684, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.060095020000000006, + "calls": 705 + }, + "rns_scale": { + "seconds": 0.21518545200000003, + "calls": 192 + }, + "other": { + "seconds": 0.061856571000000006, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 23598495, + "public_keys_bytes": 85602740, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "residue-0": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 0, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + }, + "residue-1": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 1835232, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 1, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + }, + "residue-2": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 1835232, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 2, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 591648 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "ad70ee74af27d055ac9653a58ed46b3c58381aef2de875568cdd15eeb5a9fac6", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "499ed412000271c75c1980a4f2f312ef62d5da069f7f0995957a0de77dd2ea0a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "0d8b6a02fddf6b58996da91146ad161ee88783475d87edefe93946ae3defb0c6", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "cabbd600ca70b75fa0c608bd3fc6f40dca43a695ddc8e5aeb146277790da36fa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "48c2b2c7c8fcc8c957009cf328963026b15baf03f765a35cb616ffe02e875c00", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "039e8e26c193be99f4034f6bd39c0b6e5b7b95b1b33c4cd022af0f2dbd358632", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "05e30fc3b8191e2fd0f0620ac7dbbc35ccea0a43913fc62f66575b0d44716aeb", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "e002a06dac9b5f33cecc72dd6d08914417d9882430d829967613bb5d5f661299", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_fused_32.json b/benchmarks/results/native_bfv_fused_32.json new file mode 100644 index 0000000..6f8f773 --- /dev/null +++ b/benchmarks/results/native_bfv_fused_32.json @@ -0,0 +1,562 @@ +{ + "environment": { + "utc": "2026-09-11T03:05:33.347461+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "e5c075106c91d52d927af65e3d324f516c8c53b0", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "32", + "--rns-modulus", + "--backends", + "residue", + "--residue-levels", + "0,1,2", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_fused_32.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, arithmetic context creation (reported under setup), key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "backend_variants": { + "residue-0": [ + "residue", + 0 + ], + "residue-1": [ + "residue", + 1 + ], + "residue-2": [ + "residue", + 2 + ] + }, + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "ffffffffffbc0030000004bff78002fff1004bffbc001", + "ciphertext_modulus_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "seed": 1337, + "vectors": 32, + "setup_timings": { + "key_generation_s": 4.475579569814727, + "index_encrypt_s": 0.21834237920120358, + "query_encrypt_s": 0.1041919409763068, + "public_key_export_s": 0.8049148949794471, + "residue-0_public_key_import_s": 0.8719211141578853, + "residue-0_arithmetic_context_s": 0.0016436860896646976, + "residue-1_public_key_import_s": 0.8792678711470217, + "residue-1_arithmetic_context_s": 0.0009159599430859089, + "residue-2_public_key_import_s": 0.8816855410113931, + "residue-2_arithmetic_context_s": 0.000876349164173007 + }, + "runs": [ + { + "backend": "residue-0", + "repeat": 0, + "server_compute_s": 0.401618508156389 + }, + { + "backend": "residue-1", + "repeat": 0, + "server_compute_s": 0.39036817383021116 + }, + { + "backend": "residue-2", + "repeat": 0, + "server_compute_s": 0.3826875661034137 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 0.08032198995351791 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 0.07701776688918471 + }, + { + "backend": "residue-2", + "repeat": 1, + "server_compute_s": 0.0762762411031872 + }, + { + "backend": "residue-1", + "repeat": 1, + "server_compute_s": 0.082836874993518 + }, + { + "backend": "residue-0", + "repeat": 1, + "server_compute_s": 0.08717664889991283 + }, + { + "backend": "residue-0", + "repeat": 2, + "server_compute_s": 0.08822069107554853 + }, + { + "backend": "residue-1", + "repeat": 2, + "server_compute_s": 0.08271700097247958 + }, + { + "backend": "residue-2", + "repeat": 2, + "server_compute_s": 0.0763571229763329 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 0.08028713893145323 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 0.07686260412447155 + }, + { + "backend": "residue-2", + "repeat": 3, + "server_compute_s": 0.0761273680254817 + }, + { + "backend": "residue-1", + "repeat": 3, + "server_compute_s": 0.08302503800950944 + }, + { + "backend": "residue-0", + "repeat": 3, + "server_compute_s": 0.08712895796634257 + } + ], + "summary": { + "residue-0": { + "first_search_s": 0.401618508156389, + "warm_searches_s": [ + 0.08717664889991283, + 0.08822069107554853, + 0.08712895796634257 + ], + "warm_median_s": 0.08717664889991283 + }, + "residue-1": { + "first_search_s": 0.39036817383021116, + "warm_searches_s": [ + 0.082836874993518, + 0.08271700097247958, + 0.08302503800950944 + ], + "warm_median_s": 0.082836874993518 + }, + "residue-2": { + "first_search_s": 0.3826875661034137, + "warm_searches_s": [ + 0.0762762411031872, + 0.0763571229763329, + 0.0761273680254817 + ], + "warm_median_s": 0.0762762411031872 + }, + "seal": { + "first_search_s": 0.08032198995351791, + "warm_searches_s": [ + 0.07701776688918471, + 0.08028713893145323, + 0.07686260412447155 + ], + "warm_median_s": 0.07701776688918471 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "residue-0": { + "first": 1.0, + "warm_median": 1.0 + }, + "residue-1": { + "first": 1.0288198041756118, + "warm_median": 1.0523893967117233 + }, + "residue-2": { + "first": 1.0494684011966555, + "warm_median": 1.1429069870128952 + }, + "seal": { + "first": 5.000106551005576, + "warm_median": 1.1319030974417241 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 865029, + "query_bytes": 432641, + "public_key_bytes": 39586570, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 103298, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "residue-0": { + "forward_ntt": { + "seconds": 0.035739227000000005, + "calls": 418 + }, + "inverse_ntt": { + "seconds": 0.013498248000000001, + "calls": 180 + }, + "to_residues": { + "seconds": 0.004273492, + "calls": 10 + }, + "crt_reconstruct": { + "seconds": 0.005981719000000001, + "calls": 8 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.002274541, + "calls": 399 + }, + "pointwise": { + "seconds": 0.008908133, + "calls": 35 + }, + "scale_round": { + "seconds": 0.006335980000000001, + "calls": 7 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.000614412, + "calls": 27 + }, + "automorphism": { + "seconds": 0.0008823390000000001, + "calls": 19 + }, + "add_sub": { + "seconds": 0.001745765, + "calls": 42 + }, + "wire_import": { + "seconds": 0.0013987300000000002, + "calls": 6 + }, + "wire_export": { + "seconds": 0.000160513, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.001924134, + "calls": 23 + }, + "rns_scale": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.003255571, + "calls": 1 + } + }, + "residue-1": { + "forward_ntt": { + "seconds": 0.032785657, + "calls": 418 + }, + "inverse_ntt": { + "seconds": 0.011867049000000001, + "calls": 180 + }, + "to_residues": { + "seconds": 0.004252533, + "calls": 10 + }, + "crt_reconstruct": { + "seconds": 0.006173485, + "calls": 8 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.00225163, + "calls": 399 + }, + "pointwise": { + "seconds": 0.008610774, + "calls": 35 + }, + "scale_round": { + "seconds": 0.006365468, + "calls": 7 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.000553022, + "calls": 27 + }, + "automorphism": { + "seconds": 0.0008768980000000001, + "calls": 19 + }, + "add_sub": { + "seconds": 0.0017867780000000002, + "calls": 42 + }, + "wire_import": { + "seconds": 0.00135716, + "calls": 6 + }, + "wire_export": { + "seconds": 0.000165684, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.0019463630000000002, + "calls": 23 + }, + "rns_scale": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.003454454, + "calls": 1 + } + }, + "residue-2": { + "forward_ntt": { + "seconds": 0.033162801, + "calls": 418 + }, + "inverse_ntt": { + "seconds": 0.011793217, + "calls": 180 + }, + "to_residues": { + "seconds": 0.002709368, + "calls": 4 + }, + "crt_reconstruct": { + "seconds": 0.00033013700000000004, + "calls": 2 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.002518695, + "calls": 399 + }, + "pointwise": { + "seconds": 0.008948329000000001, + "calls": 35 + }, + "scale_round": { + "seconds": 0.0012418260000000001, + "calls": 1 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.000535945, + "calls": 27 + }, + "automorphism": { + "seconds": 0.0009641210000000001, + "calls": 19 + }, + "add_sub": { + "seconds": 0.001791826, + "calls": 42 + }, + "wire_import": { + "seconds": 0.00137702, + "calls": 6 + }, + "wire_export": { + "seconds": 0.00016111400000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.001975412, + "calls": 23 + }, + "rns_scale": { + "seconds": 0.006580942, + "calls": 6 + }, + "other": { + "seconds": 0.0026012870000000003, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 737480, + "public_keys_bytes": 85603026, + "query_bytes": 368754, + "response_bytes": 102494, + "query_plus_response_bytes": 471248 + }, + "caches": { + "residue-0": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 0, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + }, + "residue-1": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 1835232, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 1, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + }, + "residue-2": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 1835232, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 2, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 591648 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 31, + "distance": 0 + }, + { + "index": 19, + "distance": 241 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "ad70ee74af27d055ac9653a58ed46b3c58381aef2de875568cdd15eeb5a9fac6", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "499ed412000271c75c1980a4f2f312ef62d5da069f7f0995957a0de77dd2ea0a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "0d8b6a02fddf6b58996da91146ad161ee88783475d87edefe93946ae3defb0c6", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "cabbd600ca70b75fa0c608bd3fc6f40dca43a695ddc8e5aeb146277790da36fa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "48c2b2c7c8fcc8c957009cf328963026b15baf03f765a35cb616ffe02e875c00", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "039e8e26c193be99f4034f6bd39c0b6e5b7b95b1b33c4cd022af0f2dbd358632", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "05e30fc3b8191e2fd0f0620ac7dbbc35ccea0a43913fc62f66575b0d44716aeb", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "e002a06dac9b5f33cecc72dd6d08914417d9882430d829967613bb5d5f661299", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_guarded_original_1024.json b/benchmarks/results/native_bfv_guarded_original_1024.json new file mode 100644 index 0000000..2f1d75b --- /dev/null +++ b/benchmarks/results/native_bfv_guarded_original_1024.json @@ -0,0 +1,191 @@ +{ + "environment": { + "utc": "2026-09-11T13:41:44.618087+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "cpu": "AMD Ryzen 7 5800X 8-Core Processor", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "pycryptodome": "3.23.0", + "msgpack": "1.1.2", + "compiler": "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0", + "revision": "0cd7b41454533ab27082667cb9cb2f3c8040e823", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_guarded.py", + "--profile", + "original", + "--json-out", + "benchmarks/results/native_bfv_guarded_original_1024.json" + ], + "profile": "original", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "vectors": 1024, + "seed": 1337, + "measurement_notes": "Within each profile all variants use identical keys and evaluated ciphertexts. Server and owner-controlled public-only verifier have independently imported setups and separate residue arithmetic contexts. Server search and verifier approval include authentication, MessagePack and arithmetic; setup import, arithmetic-context creation, query encryption and private decoding are separate. Approval independently recomputes the full result and compares exact bytes before Ed25519 signing. Server then verifier run sequentially; no network or overlap claimed. One cold trial followed by three warm trials by default. Raw Python/native client timers include ciphertext-array parsing and all distance decoding; guarded includes receipt, authentication, replay and private result checks, with alternating client order. No security-level or end-to-end constant-time claim.", + "setup_timings": { + "key_generation_s": 4.318560369079933, + "client_key_snapshot_native_import_s": 0.004531735088676214, + "encrypt_index_summary_export_auth_s": 8.078310951124877, + "server_setup_import_s": 1.7306867081206292, + "verifier_setup_import_s": 1.7689348999410868, + "server_arithmetic_context_s": 0.0012989682145416737, + "verifier_arithmetic_context_s": 0.0009495900012552738 + }, + "runs": [ + { + "repeat": 0, + "server": 2.669166201958433, + "verifier": 2.656381974928081, + "python_raw_client": 0.0393350338563323, + "native_raw_client": 0.012509942054748535, + "guarded_client": 0.016294020926579833 + }, + { + "repeat": 1, + "server": 2.3515148740261793, + "verifier": 2.3537557921372354, + "python_raw_client": 0.039415847044438124, + "native_raw_client": 0.012399370083585382, + "guarded_client": 0.01612957799807191 + }, + { + "repeat": 2, + "server": 2.353946026880294, + "verifier": 2.3622144910041243, + "python_raw_client": 0.03945690696127713, + "native_raw_client": 0.012485342100262642, + "guarded_client": 0.016256339848041534 + }, + { + "repeat": 3, + "server": 2.3585687349550426, + "verifier": 2.3510658380109817, + "python_raw_client": 0.038974247174337506, + "native_raw_client": 0.012455580988898873, + "guarded_client": 0.016008156118914485 + } + ], + "summary": { + "server": { + "first_s": 2.669166201958433, + "warm_samples_s": [ + 2.3515148740261793, + 2.353946026880294, + 2.3585687349550426 + ], + "warm_median_s": 2.353946026880294 + }, + "verifier": { + "first_s": 2.656381974928081, + "warm_samples_s": [ + 2.3537557921372354, + 2.3622144910041243, + 2.3510658380109817 + ], + "warm_median_s": 2.3537557921372354 + }, + "python_raw_client": { + "first_s": 0.0393350338563323, + "warm_samples_s": [ + 0.039415847044438124, + 0.03945690696127713, + 0.038974247174337506 + ], + "warm_median_s": 0.039415847044438124 + }, + "native_raw_client": { + "first_s": 0.012509942054748535, + "warm_samples_s": [ + 0.012399370083585382, + 0.012485342100262642, + 0.012455580988898873 + ], + "warm_median_s": 0.012455580988898873 + }, + "guarded_client": { + "first_s": 0.016294020926579833, + "warm_samples_s": [ + 0.01612957799807191, + 0.016256339848041534, + 0.016008156118914485 + ], + "warm_median_s": 0.01612957799807191 + } + }, + "query_encrypt_and_bind_samples_s": [ + 0.09995461604557931, + 0.10021196096204221, + 0.10079239308834076, + 0.10059099900536239 + ], + "sizes": { + "public_keys_bytes_per_evaluator": 85602438, + "encrypted_index_bytes_per_evaluator": 23598467, + "authenticated_setup_bytes_per_evaluator": 109205199, + "query_bytes": 368840, + "response_bytes": 102614, + "receipt_bytes": 198, + "client_response_and_receipt_bytes": 102812, + "client_query_response_receipt_bytes": 471652, + "remote_verifier_additional_query_response_input_bytes": 471454, + "private_verification_payload_bytes": 16448, + "native_locked_key_bytes": 131072, + "native_locked_scratch_bytes_per_active_decode": 196608 + }, + "whole_process_peak_rss_kib_linux": 1094104, + "noise_bound": { + "encrypted": 344085, + "difference": 689382, + "squared": 6077954481171225104, + "key_switch": 1108307719766016, + "tile": 1670999460573335352275202771320, + "merged": 855551723813548266710148620588680, + "compacted": 69633, + "original_rounding_error_numerator": 56070293323568512755383010147599750792, + "terminal_rounding_error_numerator": 8856539137, + "original_modulus": 1532495540865518635130821056977027158796330141975560193, + "terminal_modulus": 1125899906842597, + "sufficient": true + }, + "identical_ciphertexts_within_profile": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_guarded.py": "1c82dbe809674bc70d6946b0f4a7827c0d8d2c8ce715dc1f2d25356a7a73aa79", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_private.py": "b1291e6d017be84638b054fb84e651734a44b2d54d601eb50dae2e9d2016bdfa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp": "7b48d92e526147d97d9401e868c8d3e9dbf0b8ba567a67e05047e01f2a371fc0", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_decoder.h": "66aceec7cc8452dabdde0061d347195e49a9500fba7bcbbe805c967b7171d767", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ee3f7371f92ba9fa3ebb8037906fc23c5f7832f96908d5879f1a4326847ffb53", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } +} diff --git a/benchmarks/results/native_bfv_guarded_review_1024.json b/benchmarks/results/native_bfv_guarded_review_1024.json new file mode 100644 index 0000000..6d45f94 --- /dev/null +++ b/benchmarks/results/native_bfv_guarded_review_1024.json @@ -0,0 +1,191 @@ +{ + "environment": { + "utc": "2026-09-11T13:43:35.312233+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "cpu": "AMD Ryzen 7 5800X 8-Core Processor", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "pycryptodome": "3.23.0", + "msgpack": "1.1.2", + "compiler": "c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0", + "revision": "0cd7b41454533ab27082667cb9cb2f3c8040e823", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_guarded.py", + "--profile", + "review", + "--json-out", + "benchmarks/results/native_bfv_guarded_review_1024.json" + ], + "profile": "review", + "config": { + "embed_len": 512, + "poly_modulus_degree": 16384, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "vectors": 1024, + "seed": 1337, + "measurement_notes": "Within each profile all variants use identical keys and evaluated ciphertexts. Server and owner-controlled public-only verifier have independently imported setups and separate residue arithmetic contexts. Server search and verifier approval include authentication, MessagePack and arithmetic; setup import, arithmetic-context creation, query encryption and private decoding are separate. Approval independently recomputes the full result and compares exact bytes before Ed25519 signing. Server then verifier run sequentially; no network or overlap claimed. One cold trial followed by three warm trials by default. Raw Python/native client timers include ciphertext-array parsing and all distance decoding; guarded includes receipt, authentication, replay and private result checks, with alternating client order. No security-level or end-to-end constant-time claim.", + "setup_timings": { + "key_generation_s": 8.871012358926237, + "client_key_snapshot_native_import_s": 0.0067274910397827625, + "encrypt_index_summary_export_auth_s": 9.416916926158592, + "server_setup_import_s": 3.3731832071207464, + "verifier_setup_import_s": 3.446248749969527, + "server_arithmetic_context_s": 0.0023073479533195496, + "verifier_arithmetic_context_s": 0.001975191989913583 + }, + "runs": [ + { + "repeat": 0, + "server": 3.071234895149246, + "verifier": 3.0581927029415965, + "python_raw_client": 0.07951001706533134, + "native_raw_client": 0.024246918968856335, + "guarded_client": 0.02819207194261253 + }, + { + "repeat": 1, + "server": 2.4405403789132833, + "verifier": 2.437672710046172, + "python_raw_client": 0.0788352529052645, + "native_raw_client": 0.024385120952501893, + "guarded_client": 0.028834844008088112 + }, + { + "repeat": 2, + "server": 2.4386640209704638, + "verifier": 2.435292261186987, + "python_raw_client": 0.08160242205485702, + "native_raw_client": 0.025453513953834772, + "guarded_client": 0.028343144105747342 + }, + { + "repeat": 3, + "server": 2.4315330830868334, + "verifier": 2.432053772965446, + "python_raw_client": 0.080289953853935, + "native_raw_client": 0.024648976977914572, + "guarded_client": 0.028425215976312757 + } + ], + "summary": { + "server": { + "first_s": 3.071234895149246, + "warm_samples_s": [ + 2.4405403789132833, + 2.4386640209704638, + 2.4315330830868334 + ], + "warm_median_s": 2.4386640209704638 + }, + "verifier": { + "first_s": 3.0581927029415965, + "warm_samples_s": [ + 2.437672710046172, + 2.435292261186987, + 2.432053772965446 + ], + "warm_median_s": 2.435292261186987 + }, + "python_raw_client": { + "first_s": 0.07951001706533134, + "warm_samples_s": [ + 0.0788352529052645, + 0.08160242205485702, + 0.080289953853935 + ], + "warm_median_s": 0.080289953853935 + }, + "native_raw_client": { + "first_s": 0.024246918968856335, + "warm_samples_s": [ + 0.024385120952501893, + 0.025453513953834772, + 0.024648976977914572 + ], + "warm_median_s": 0.024648976977914572 + }, + "guarded_client": { + "first_s": 0.02819207194261253, + "warm_samples_s": [ + 0.028834844008088112, + 0.028343144105747342, + 0.028425215976312757 + ], + "warm_median_s": 0.028425215976312757 + } + }, + "query_encrypt_and_bind_samples_s": [ + 0.20502086984924972, + 0.2047637349460274, + 0.20417200191877782, + 0.20610628393478692 + ], + "sizes": { + "public_keys_bytes_per_evaluator": 171205156, + "encrypted_index_bytes_per_evaluator": 23595715, + "authenticated_setup_bytes_per_evaluator": 194805166, + "query_bytes": 737480, + "response_bytes": 205018, + "receipt_bytes": 198, + "client_response_and_receipt_bytes": 205216, + "client_query_response_receipt_bytes": 942696, + "remote_verifier_additional_query_response_input_bytes": 942498, + "private_verification_payload_bytes": 16448, + "native_locked_key_bytes": 262144, + "native_locked_scratch_bytes_per_active_decode": 393216 + }, + "whole_process_peak_rss_kib_linux": 1985372, + "noise_bound": { + "encrypted": 688149, + "difference": 1427785, + "squared": 52059769849576678535, + "key_switch": 2216615439532032, + "tile": 28621377361628378284959515398718, + "merged": 14654145209153730814589761537631682, + "compacted": 73729, + "original_rounding_error_numerator": 960388714572308056395769201895141795266, + "terminal_rounding_error_numerator": 9124978689, + "original_modulus": 1532495540865191964058577030327937992314109603578642433, + "terminal_modulus": 1125899906842597, + "sufficient": true + }, + "identical_ciphertexts_within_profile": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_guarded.py": "1c82dbe809674bc70d6946b0f4a7827c0d8d2c8ce715dc1f2d25356a7a73aa79", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_private.py": "b1291e6d017be84638b054fb84e651734a44b2d54d601eb50dae2e9d2016bdfa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp": "7b48d92e526147d97d9401e868c8d3e9dbf0b8ba567a67e05047e01f2a371fc0", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_decoder.h": "66aceec7cc8452dabdde0061d347195e49a9500fba7bcbbe805c967b7171d767", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ee3f7371f92ba9fa3ebb8037906fc23c5f7832f96908d5879f1a4326847ffb53", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } +} diff --git a/benchmarks/results/native_bfv_nitro_audit_1024.json b/benchmarks/results/native_bfv_nitro_audit_1024.json new file mode 100644 index 0000000..fe4a74d --- /dev/null +++ b/benchmarks/results/native_bfv_nitro_audit_1024.json @@ -0,0 +1,164 @@ +{ + "mode": "local", + "hardware_attestation": false, + "measurement_notes": "All variants evaluate identical ciphertexts; exactly one evaluation produces a protocol response. Local mode substitutes a test CA and includes real certificate, owner-signature and receipt checks, but no NSM, vsock, hardware isolation or EC2. AWS protocol timers include network/relay; the reference runs locally and ratios are meaningful only on matched hardware. One cold and remaining warm trials; no tail-latency claim. No production-security certification.", + "environment": { + "utc": "2026-09-15T17:55:52.950792+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "cpu": "AMD Ryzen 7 5800X 8-Core Processor", + "versions": { + "gmpy2": "2.3.0", + "pycryptodome": "3.23.0", + "cbor2": "6.1.4", + "pyOpenSSL": "26.4.0", + "cryptography": "50.0.1" + }, + "revision": "46b637b604e1c2835653140610b7b2396f10b284" + }, + "config": { + "embed_len": 512, + "poly_modulus_degree": 16384, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 8.675407716073096, + "index_encrypt_export_s": 9.33344871387817, + "registration_encode_s": 0.3417240921407938, + "baseline_import_s": 3.3312883710023016, + "protocol_import_s": 3.5784192520659417 + }, + "enrollment": [ + { + "issue_or_rpc_s": 0.0022853699047118425, + "client_verify_s": 0.0075106131844222546, + "document_bytes": 1936, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.0021545880008488894, + "client_verify_s": 0.005094030173495412, + "document_bytes": 1936, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.002078167162835598, + "client_verify_s": 0.00487562594935298, + "document_bytes": 1936, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.002088826848194003, + "client_verify_s": 0.00495729804970324, + "document_bytes": 1936, + "possession_bytes": 64, + "challenge_bytes": 134 + } + ], + "runs": [ + { + "repeat": 0, + "query_encrypt_sign_s": 0.20626455708406866, + "baseline_local_search_s": 3.060150825884193, + "protocol_search_or_rpc_s": 3.06122068897821, + "client_finish_s": 0.028632056899368763 + }, + { + "repeat": 1, + "query_encrypt_sign_s": 0.20665679895319045, + "baseline_local_search_s": 2.4312804299406707, + "protocol_search_or_rpc_s": 2.436255524866283, + "client_finish_s": 0.02823221101425588 + }, + { + "repeat": 2, + "query_encrypt_sign_s": 0.20398432505317032, + "baseline_local_search_s": 2.432181579992175, + "protocol_search_or_rpc_s": 2.435416348045692, + "client_finish_s": 0.028609218075871468 + }, + { + "repeat": 3, + "query_encrypt_sign_s": 0.2063854408916086, + "baseline_local_search_s": 2.4394479650072753, + "protocol_search_or_rpc_s": 2.4388709040358663, + "client_finish_s": 0.02783559402450919 + } + ], + "summary": { + "query_encrypt_sign_s": { + "first_s": 0.20626455708406866, + "warm_median_s": 0.2063854408916086, + "warm_samples_s": [ + 0.20665679895319045, + 0.20398432505317032, + 0.2063854408916086 + ] + }, + "baseline_local_search_s": { + "first_s": 3.060150825884193, + "warm_median_s": 2.432181579992175, + "warm_samples_s": [ + 2.4312804299406707, + 2.432181579992175, + 2.4394479650072753 + ] + }, + "protocol_search_or_rpc_s": { + "first_s": 3.06122068897821, + "warm_median_s": 2.436255524866283, + "warm_samples_s": [ + 2.436255524866283, + 2.435416348045692, + 2.4388709040358663 + ] + }, + "client_finish_s": { + "first_s": 0.028632056899368763, + "warm_median_s": 0.02823221101425588, + "warm_samples_s": [ + 0.02823221101425588, + 0.028609218075871468, + 0.02783559402450919 + ] + } + }, + "sizes": { + "setup_bytes": 194804883, + "registration_bytes": 194805024, + "query_bytes": 737582, + "response_bytes": 205018, + "receipt_bytes": 198, + "query_response_receipt_bytes": 942798 + }, + "whole_process_peak_rss_kib_linux": 2320364, + "identical_ciphertexts": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_nitro.py": "6189c69c762e11d7b3cffcb66a695e7e7306d8350f0cb4245b2c89542611aefe", + "experiments/bfv_nitro/__init__.py": "96f217ab43bd80b51d6b928cc128a7f7d6fb9c8bce318267ac157436bb49df75", + "experiments/bfv_nitro/client.py": "5a5d68dbde3f07f053e0d67b3ab22a1da1cc493cf631b540237408b0f5f7fe6a", + "experiments/bfv_nitro/relay.py": "113c99417e3ff4b9f7769084ccf8bb19225d351c2b545786980e4a7e1939e00f", + "experiments/bfv_nitro/service.py": "a4b322e678d9f49a16d44f76f53e9ea3fe2094ed213cb6f133aef21643c9e701", + "experiments/bfv_nitro/transport.py": "1fbb2767aa59209d0bdadd17b7aaafbfeb8f11a72e64cbb5f7997368a1bb36d5", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/crypto/bfv_attested_client.py": "d3bed7a596fd19f834ea308fc332f0a28109a1136c7dd63026819b4cc106c7de", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_nitro.py": "791107dd7daaea99b07cb9cc70116ec71fe932c0ece608a9bd65ce8a7aad6660", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0" + } +} diff --git a/benchmarks/results/native_bfv_nitro_local_1024.json b/benchmarks/results/native_bfv_nitro_local_1024.json new file mode 100644 index 0000000..e446a18 --- /dev/null +++ b/benchmarks/results/native_bfv_nitro_local_1024.json @@ -0,0 +1,164 @@ +{ + "mode": "local", + "hardware_attestation": false, + "measurement_notes": "All variants evaluate identical ciphertexts; exactly one evaluation produces a protocol response. Local mode substitutes a test CA and includes real certificate, owner-signature and receipt checks, but no NSM, vsock, hardware isolation or EC2. AWS protocol timers include network/relay; the reference runs locally and ratios are meaningful only on matched hardware. One cold and remaining warm trials; no tail-latency claim. No production-security certification.", + "environment": { + "utc": "2026-09-13T03:34:41.769069+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "cpu": "AMD Ryzen 7 5800X 8-Core Processor", + "versions": { + "gmpy2": "2.3.0", + "pycryptodome": "3.23.0", + "cbor2": "6.1.4", + "pyOpenSSL": "26.4.0", + "cryptography": "50.0.1" + }, + "revision": "46b637b604e1c2835653140610b7b2396f10b284" + }, + "config": { + "embed_len": 512, + "poly_modulus_degree": 16384, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 8.720161478966475, + "index_encrypt_export_s": 9.22493529994972, + "registration_encode_s": 0.33406802802346647, + "baseline_import_s": 3.288292540004477, + "protocol_import_s": 3.5667430909816176 + }, + "enrollment": [ + { + "issue_or_rpc_s": 0.002338145859539509, + "client_verify_s": 0.00759821105748415, + "document_bytes": 1938, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.002176213078200817, + "client_verify_s": 0.0049460879527032375, + "document_bytes": 1938, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.0021135020069777966, + "client_verify_s": 0.005014879861846566, + "document_bytes": 1938, + "possession_bytes": 64, + "challenge_bytes": 134 + }, + { + "issue_or_rpc_s": 0.002089841989800334, + "client_verify_s": 0.004991688998416066, + "document_bytes": 1938, + "possession_bytes": 64, + "challenge_bytes": 134 + } + ], + "runs": [ + { + "repeat": 0, + "query_encrypt_sign_s": 0.2101371439639479, + "baseline_local_search_s": 3.065155303804204, + "protocol_search_or_rpc_s": 3.0666417400352657, + "client_finish_s": 0.028333893977105618 + }, + { + "repeat": 1, + "query_encrypt_sign_s": 0.20469022588804364, + "baseline_local_search_s": 2.4343131280038506, + "protocol_search_or_rpc_s": 2.438590140081942, + "client_finish_s": 0.028182381065562367 + }, + { + "repeat": 2, + "query_encrypt_sign_s": 0.2030612831003964, + "baseline_local_search_s": 2.439071057131514, + "protocol_search_or_rpc_s": 2.4393830781336874, + "client_finish_s": 0.028104739962145686 + }, + { + "repeat": 3, + "query_encrypt_sign_s": 0.20315458718687296, + "baseline_local_search_s": 2.4386219407897443, + "protocol_search_or_rpc_s": 2.4390968158841133, + "client_finish_s": 0.0280205390881747 + } + ], + "summary": { + "query_encrypt_sign_s": { + "first_s": 0.2101371439639479, + "warm_median_s": 0.20315458718687296, + "warm_samples_s": [ + 0.20469022588804364, + 0.2030612831003964, + 0.20315458718687296 + ] + }, + "baseline_local_search_s": { + "first_s": 3.065155303804204, + "warm_median_s": 2.4386219407897443, + "warm_samples_s": [ + 2.4343131280038506, + 2.439071057131514, + 2.4386219407897443 + ] + }, + "protocol_search_or_rpc_s": { + "first_s": 3.0666417400352657, + "warm_median_s": 2.4390968158841133, + "warm_samples_s": [ + 2.438590140081942, + 2.4393830781336874, + 2.4390968158841133 + ] + }, + "client_finish_s": { + "first_s": 0.028333893977105618, + "warm_median_s": 0.028104739962145686, + "warm_samples_s": [ + 0.028182381065562367, + 0.028104739962145686, + 0.0280205390881747 + ] + } + }, + "sizes": { + "setup_bytes": 194805676, + "registration_bytes": 194805817, + "query_bytes": 737582, + "response_bytes": 205018, + "receipt_bytes": 198, + "query_response_receipt_bytes": 942798 + }, + "whole_process_peak_rss_kib_linux": 2319872, + "identical_ciphertexts": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_nitro.py": "6189c69c762e11d7b3cffcb66a695e7e7306d8350f0cb4245b2c89542611aefe", + "experiments/bfv_nitro/__init__.py": "96f217ab43bd80b51d6b928cc128a7f7d6fb9c8bce318267ac157436bb49df75", + "experiments/bfv_nitro/client.py": "8dcefbf02435d1ab4ffa73d860a025b16e89cba7f52d8ea70444947b5316b48e", + "experiments/bfv_nitro/relay.py": "a773e845ef680cd3f9674728d2a781d1c8cd5d80385c942439d1f3031ab10e59", + "experiments/bfv_nitro/service.py": "1354ca96c4c9ac65753dcfaf2cf86b9993a3cb35f4f9e7c3e165be56f50e3147", + "experiments/bfv_nitro/transport.py": "ede43317af73b53fa19fb95a84b147dcd9e47e3032386e67afa5daefed0e3b2a", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/crypto/bfv_attested_client.py": "cbab29f860b07d44264203c6689fece4044a08b76c7564a909eacff973993dcf", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_nitro.py": "94ac7be118ae20d4e6c58cc903cf26bb98e66d3928dd9d143616c5b479b5a654", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0" + } +} diff --git a/benchmarks/results/native_bfv_parameter_estimates.json b/benchmarks/results/native_bfv_parameter_estimates.json new file mode 100644 index 0000000..1c7ccd4 --- /dev/null +++ b/benchmarks/results/native_bfv_parameter_estimates.json @@ -0,0 +1,521 @@ +{ + "utc": "2026-09-11T13:06:59.442710+00:00", + "sage": "10.9", + "python": "3.14.6", + "estimator_revision": "53da5982597709ba0fdf94ea37a84d822310fd84", + "runner_sha256": "8161c495e2b6932a5321b19dfa4da56f25b09934ae6df0b98a36d53e64425b7b", + "manifest": { + "schema": 1, + "profile": "original", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "q": "1532495540865518635130821056977027158796330141975560193", + "q_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "secret": { + "distribution": "uniform", + "inclusive_bounds": [ + -1, + 1 + ] + }, + "error": { + "distribution": "centered_binomial", + "eta": 21, + "variance": 10.5 + }, + "ephemeral_u": { + "distribution": "uniform", + "inclusive_bounds": [ + -1, + 1 + ] + }, + "evaluation_keys": { + "galois_exponents": [ + 385, + 769, + 1537, + 2657, + 3073, + 4097, + 5953, + 6145, + 6561, + 8193, + 10241, + 12289, + 13313, + 14529, + 14849, + 15617, + 16001 + ], + "digits_per_key": 6, + "key_switch_pairs": 108, + "note": "Encryptions of gadget-scaled s^2 and automorphic secrets. Ordinary LWE estimates do not justify these circular/key-dependent-message assumptions." + }, + "sdk_revision": "0cd7b41454533ab27082667cb9cb2f3c8040e823", + "worst_supported_hamming_bound": { + "encrypted": 344085, + "difference": 689382, + "squared": 6077954481171225104, + "key_switch": 1108307719766016, + "tile": 1670999460573335352275202771320, + "merged": 855551723813548266710148620588680, + "compacted": 69633, + "original_rounding_error_numerator": 56070293323568512755383010147599750792, + "terminal_rounding_error_numerator": 8856539137, + "original_modulus": 1532495540865518635130821056977027158796330141975560193, + "terminal_modulus": 1125899906842597, + "sufficient": true + }, + "source_sha256": { + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } + }, + "command": [ + "benchmarks/bfv_parameter_audit.py", + "estimate", + "--manifest", + "/tmp/xtrace-bfv-parameter-manifest.json", + "--estimator", + "/tmp/xtrace-bfv-lattice-estimator", + "--json-out", + "benchmarks/results/native_bfv_parameter_estimates.json" + ], + "interpretation": "Heuristic generic LWE attack estimates with Gaussian-shape approximation for centered binomial errors. Reports m=N and unbounded independent samples; RLWE structure, evaluation-key assumptions, decryption failures, implementation leakage and protocol attacks are not certified. ADPS16-quantum is a quantum sieving cost sensitivity model, not a complete quantum security analysis. Unlisted attacks are not evaluated.", + "runs": [ + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "primal_usvp", + "seconds": 0.052037457935512066, + "log2_rop": 154.8562154453958, + "cost": { + "rop": "4.13395613507757e46", + "red": "4.13395613507757e46", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "primal_bdd", + "seconds": 2.4931123971473426, + "log2_rop": 154.3993899051486, + "cost": { + "rop": "3.01194941468505e46", + "red": "2.88343174765430e46", + "svp": "1.28517667030745e45", + "beta": "427", + "eta": "460", + "d": "16145", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "dual", + "seconds": 0.03983351797796786, + "log2_rop": 156.18958860022911, + "cost": { + "rop": "1.04172042412270e47", + "mem": "3729007849560365844932083519906", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "dual_hybrid", + "seconds": 4.287429159972817, + "log2_rop": 155.04029557642883, + "cost": { + "rop": "4.69655600047778e46", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)", + "red": "4.69655589846026e46", + "guess": "1.02017520797435e39", + "beta": "426", + "p": "3", + "zeta": "0", + "t": "70", + "beta_": "443", + "N": "3.12173363136213e27", + "m": "8192" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "primal_usvp", + "seconds": 0.03931264718994498, + "log2_rop": 125.26799999999999, + "cost": { + "rop": "5.12183397729253e37", + "red": "5.12183397729253e37", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "primal_bdd", + "seconds": 2.626561006065458, + "log2_rop": 125.28842587069742, + "cost": { + "rop": "5.19486536675701e37", + "red": "5.12183397729253e37", + "svp": "7.30313894644757e35", + "beta": "429", + "eta": "407", + "d": "16231", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "dual", + "seconds": 0.028203140944242477, + "log2_rop": 125.5600000000002, + "cost": { + "rop": "6.27084773423208e37", + "mem": "311087831586185034754193559976", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "dual_hybrid", + "seconds": 2.994258353020996, + "log2_rop": 124.69359450053989, + "cost": { + "rop": "3.3836789e37*e^(-5) + 3.41683478187330e37", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)", + "red": "3.41683478187330e37", + "guess": "3.3836789e37*e^(-5)", + "beta": "427", + "p": "3", + "zeta": "10", + "t": "40", + "beta_": "427", + "N": "1.42331887192754e26", + "m": "8192" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "primal_usvp", + "seconds": 0.039275356102734804, + "log2_rop": 113.685, + "cost": { + "rop": "1.66953318958631e34", + "red": "1.66953318958631e34", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "primal_bdd", + "seconds": 2.6875910649541765, + "log2_rop": 113.7151575146729, + "cost": { + "rop": "1.70479975216603e34", + "red": "1.66953318958631e34", + "svp": "3.52665625797147e32", + "beta": "429", + "eta": "407", + "d": "16231", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "dual", + "seconds": 0.023456628900021315, + "log2_rop": 113.9500000006802, + "cost": { + "rop": "2.00617126743433e34", + "mem": "311087831586185034754193559976", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "dual_hybrid", + "seconds": 1.8661667311098427, + "log2_rop": 113.42000129809128, + "cost": { + "rop": "1.38938465805634e34", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=8192, tag=None)", + "red": "1.38938340793262e34", + "guess": "1.25012371911589e28", + "beta": "428", + "p": "3", + "zeta": "0", + "t": "40", + "beta_": "428", + "N": "1.03040264691278e25", + "m": "8192" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "primal_usvp", + "seconds": 0.03961539291776717, + "log2_rop": 154.8562154453958, + "cost": { + "rop": "4.13395613507757e46", + "red": "4.13395613507757e46", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "primal_bdd", + "seconds": 2.544985809130594, + "log2_rop": 154.3993899051486, + "cost": { + "rop": "3.01194941468505e46", + "red": "2.88343174765430e46", + "svp": "1.28517667030745e45", + "beta": "427", + "eta": "460", + "d": "16145", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "dual", + "seconds": 0.037890825886279345, + "log2_rop": 156.18958860022911, + "cost": { + "rop": "1.04172042412270e47", + "mem": "3729007849560365844932083519906", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "dual_hybrid", + "seconds": 4.301755240187049, + "log2_rop": 155.04029557642883, + "cost": { + "rop": "4.69655600047778e46", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "4.69655589846026e46", + "guess": "1.02017520797435e39", + "beta": "426", + "p": "3", + "zeta": "0", + "t": "70", + "beta_": "443", + "N": "3.12173363136213e27", + "m": "8192" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "primal_usvp", + "seconds": 0.03921360499225557, + "log2_rop": 125.26799999999999, + "cost": { + "rop": "5.12183397729253e37", + "red": "5.12183397729253e37", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "primal_bdd", + "seconds": 2.647876218194142, + "log2_rop": 125.28842587069742, + "cost": { + "rop": "5.19486536675701e37", + "red": "5.12183397729253e37", + "svp": "7.30313894644757e35", + "beta": "429", + "eta": "407", + "d": "16231", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "dual", + "seconds": 0.026285489089787006, + "log2_rop": 125.5600000000002, + "cost": { + "rop": "6.27084773423208e37", + "mem": "311087831586185034754193559976", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "dual_hybrid", + "seconds": 2.982937719207257, + "log2_rop": 124.69359450053989, + "cost": { + "rop": "3.3836789e37*e^(-5) + 3.41683478187330e37", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "3.41683478187330e37", + "guess": "3.3836789e37*e^(-5)", + "beta": "427", + "p": "3", + "zeta": "10", + "t": "40", + "beta_": "427", + "N": "1.42331887192754e26", + "m": "8192" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "primal_usvp", + "seconds": 0.03953855112195015, + "log2_rop": 113.685, + "cost": { + "rop": "1.66953318958631e34", + "red": "1.66953318958631e34", + "delta": "1.00379266173102", + "beta": "429", + "d": "15759", + "tag": "usvp", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "primal_bdd", + "seconds": 2.6987395398318768, + "log2_rop": 113.7151575146729, + "cost": { + "rop": "1.70479975216603e34", + "red": "1.66953318958631e34", + "svp": "3.52665625797147e32", + "beta": "429", + "eta": "407", + "d": "16231", + "tag": "bdd", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "dual", + "seconds": 0.02442668005824089, + "log2_rop": 113.9500000006802, + "cost": { + "rop": "2.00617126743433e34", + "mem": "311087831586185034754193559976", + "m": "8162", + "beta": "430", + "d": "16354", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "dual_hybrid", + "seconds": 1.8888231310993433, + "log2_rop": 113.42000129809128, + "cost": { + "rop": "1.38938465805634e34", + "problem": "LWEParameters(n=8192, q=1532495540865518635130821056977027158796330141975560193, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "1.38938340793262e34", + "guess": "1.25012371911589e28", + "beta": "428", + "p": "3", + "zeta": "0", + "t": "40", + "beta_": "428", + "N": "1.03040264691278e25", + "m": "8192" + } + } + ], + "complete": true +} diff --git a/benchmarks/results/native_bfv_residue_1024.json b/benchmarks/results/native_bfv_residue_1024.json new file mode 100644 index 0000000..fc3be65 --- /dev/null +++ b/benchmarks/results/native_bfv_residue_1024.json @@ -0,0 +1,407 @@ +{ + "environment": { + "utc": "2026-09-11T02:22:03.335196+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "28170049a93a15b1a06a42f5ae89d74e80f50be2", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--rns-modulus", + "--backends", + "native,residue", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_residue_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "ffffffffffbc0030000004bff78002fff1004bffbc001", + "ciphertext_modulus_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.5589400180615485, + "index_encrypt_s": 6.888210566015914, + "query_encrypt_s": 0.10452251299284399, + "public_key_export_s": 0.8208927081432194, + "native_public_key_import_s": 0.8457178140524775, + "residue_public_key_import_s": 0.8620186080224812 + }, + "runs": [ + { + "backend": "native", + "repeat": 0, + "server_compute_s": 5.6354313138872385 + }, + { + "backend": "residue", + "repeat": 0, + "server_compute_s": 3.1145015170332044 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 2.4679608230944723 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 2.463819060008973 + }, + { + "backend": "residue", + "repeat": 1, + "server_compute_s": 2.8064445089548826 + }, + { + "backend": "native", + "repeat": 1, + "server_compute_s": 5.272516414988786 + }, + { + "backend": "native", + "repeat": 2, + "server_compute_s": 5.297504244139418 + }, + { + "backend": "residue", + "repeat": 2, + "server_compute_s": 2.8145669419318438 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 2.4813549709506333 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 2.4831731480080634 + }, + { + "backend": "residue", + "repeat": 3, + "server_compute_s": 2.8025826739612967 + }, + { + "backend": "native", + "repeat": 3, + "server_compute_s": 5.278038259129971 + } + ], + "summary": { + "native": { + "first_search_s": 5.6354313138872385, + "warm_searches_s": [ + 5.272516414988786, + 5.297504244139418, + 5.278038259129971 + ], + "warm_median_s": 5.278038259129971 + }, + "residue": { + "first_search_s": 3.1145015170332044, + "warm_searches_s": [ + 2.8064445089548826, + 2.8145669419318438, + 2.8025826739612967 + ], + "warm_median_s": 2.8064445089548826 + }, + "seal": { + "first_search_s": 2.4679608230944723, + "warm_searches_s": [ + 2.463819060008973, + 2.4813549709506333, + 2.4831731480080634 + ], + "warm_median_s": 2.4813549709506333 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "native": { + "first": 1.0, + "warm_median": 1.0 + }, + "residue": { + "first": 1.809416782450441, + "warm_median": 1.8806850597931501 + }, + "seal": { + "first": 2.283436293296264, + "warm_median": 2.1270790841779075 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 27680261, + "query_bytes": 432638, + "public_key_bytes": 39584794, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 105892, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "native": { + "forward_ntt": { + "seconds": 1.580388313, + "calls": 18280 + }, + "inverse_ntt": { + "seconds": 0.597422036, + "calls": 7480 + }, + "to_residues": { + "seconds": 0.131717274, + "calls": 256 + }, + "crt_reconstruct": { + "seconds": 0.734090161, + "calls": 1726 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.063746828, + "calls": 703 + }, + "pointwise": { + "seconds": 0.45272087000000005, + "calls": 895 + }, + "scale_round": { + "seconds": 0.16394777900000002, + "calls": 193 + }, + "mod_q": { + "seconds": 0.476317101, + "calls": 1534 + }, + "buffers": { + "seconds": 0.023535914, + "calls": 1023 + }, + "automorphism": { + "seconds": 0.365672055, + "calls": 639 + }, + "add_sub": { + "seconds": 0.187055671, + "calls": 767 + }, + "wire_import": { + "seconds": 0.029517076000000003, + "calls": 130 + }, + "wire_export": { + "seconds": 0.00015730300000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.5200640830000001, + "calls": 1 + } + }, + "residue": { + "forward_ntt": { + "seconds": 1.211404176, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.478378373, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.140344466, + "calls": 320 + }, + "crt_reconstruct": { + "seconds": 0.17903116700000002, + "calls": 194 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.081616798, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.28731105500000004, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.162540489, + "calls": 193 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.020592492, + "calls": 895 + }, + "automorphism": { + "seconds": 0.031140883, + "calls": 639 + }, + "add_sub": { + "seconds": 0.05861510600000001, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.029655261000000002, + "calls": 130 + }, + "wire_export": { + "seconds": 0.000174463, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.05888882300000001, + "calls": 705 + }, + "other": { + "seconds": 0.082117907, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 23598495, + "public_keys_bytes": 85603326, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "native": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 0, + "switch_keys": 18, + "key_payload_bytes": 56623104, + "server_plan_bytes": 655360 + }, + "residue": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "846b003afc224e40d2c5edea25ffd76fc2235c343fb6f37dcfe5dffef4a56219", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "499ed412000271c75c1980a4f2f312ef62d5da069f7f0995957a0de77dd2ea0a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "0d8b6a02fddf6b58996da91146ad161ee88783475d87edefe93946ae3defb0c6", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "f76e007558fbd1f6dd6ed6929770cddc6573568fc1bec9af2b6eca31afbd24e8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "1f468bf78fba83753839eddda17b1709f05abfb9b75b4950b7a782065268b15f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "48c2b2c7c8fcc8c957009cf328963026b15baf03f765a35cb616ffe02e875c00", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "2b6e18633fd8e07e7ec5527b7d4ccba2de5a4aa866453e4c32a4e48bc731c8a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "a2fe4ca8273a3f051ba44b0a1614bc19ce757939eeb37fc57835285392682356", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "99b513895a7baca7eebb65f5a1cb3a7d10b297b14479a327269828aa1f91058f", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "fa0315210c10b41cae83fad5fb4bf493354d76837cd898a7410d32ca623e92a5", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "4ff544e8e7371aa0800c7076d1fde9f9ba4bc20143444596618be5ae1cedad47", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "d9505ac8de6f7c356512111eee9e3d8ae19dd046034a81658b021db4596ecea8", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_residue_32.json b/benchmarks/results/native_bfv_residue_32.json new file mode 100644 index 0000000..ac3e2c3 --- /dev/null +++ b/benchmarks/results/native_bfv_residue_32.json @@ -0,0 +1,407 @@ +{ + "environment": { + "utc": "2026-09-11T02:19:09.439953+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "28170049a93a15b1a06a42f5ae89d74e80f50be2", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "32", + "--rns-modulus", + "--backends", + "native,residue", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_residue_32.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "ffffffffffbc0030000004bff78002fff1004bffbc001", + "ciphertext_modulus_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "seed": 1337, + "vectors": 32, + "setup_timings": { + "key_generation_s": 4.348404427990317, + "index_encrypt_s": 0.21356452419422567, + "query_encrypt_s": 0.10177573701366782, + "public_key_export_s": 0.7944609080441296, + "native_public_key_import_s": 0.8522930489853024, + "residue_public_key_import_s": 0.8435139840003103 + }, + "runs": [ + { + "backend": "native", + "repeat": 0, + "server_compute_s": 0.5181663730181754 + }, + { + "backend": "residue", + "repeat": 0, + "server_compute_s": 0.397010528948158 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 0.07721250806935132 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 0.07963385712355375 + }, + { + "backend": "residue", + "repeat": 1, + "server_compute_s": 0.0875147629994899 + }, + { + "backend": "native", + "repeat": 1, + "server_compute_s": 0.15932174399495125 + }, + { + "backend": "native", + "repeat": 2, + "server_compute_s": 0.15907482896000147 + }, + { + "backend": "residue", + "repeat": 2, + "server_compute_s": 0.08726914902217686 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 0.07568373787216842 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 0.07749695307575166 + }, + { + "backend": "residue", + "repeat": 3, + "server_compute_s": 0.08695215196348727 + }, + { + "backend": "native", + "repeat": 3, + "server_compute_s": 0.157100300071761 + } + ], + "summary": { + "native": { + "first_search_s": 0.5181663730181754, + "warm_searches_s": [ + 0.15932174399495125, + 0.15907482896000147, + 0.157100300071761 + ], + "warm_median_s": 0.15907482896000147 + }, + "residue": { + "first_search_s": 0.397010528948158, + "warm_searches_s": [ + 0.0875147629994899, + 0.08726914902217686, + 0.08695215196348727 + ], + "warm_median_s": 0.08726914902217686 + }, + "seal": { + "first_search_s": 0.07721250806935132, + "warm_searches_s": [ + 0.07963385712355375, + 0.07568373787216842, + 0.07749695307575166 + ], + "warm_median_s": 0.07749695307575166 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "native": { + "first": 1.0, + "warm_median": 1.0 + }, + "residue": { + "first": 1.3051703550306546, + "warm_median": 1.8228071516954671 + }, + "seal": { + "first": 6.71091233758091, + "warm_median": 2.052659138798775 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 865128, + "query_bytes": 432548, + "public_key_bytes": 39585389, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 103294, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "native": { + "forward_ntt": { + "seconds": 0.047128805, + "calls": 548 + }, + "inverse_ntt": { + "seconds": 0.017303183, + "calls": 226 + }, + "to_residues": { + "seconds": 0.004098931, + "calls": 8 + }, + "crt_reconstruct": { + "seconds": 0.021912418000000003, + "calls": 52 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.001514979, + "calls": 21 + }, + "pointwise": { + "seconds": 0.012786347, + "calls": 27 + }, + "scale_round": { + "seconds": 0.0062369140000000005, + "calls": 7 + }, + "mod_q": { + "seconds": 0.014188331, + "calls": 46 + }, + "buffers": { + "seconds": 0.000676216, + "calls": 31 + }, + "automorphism": { + "seconds": 0.01051699, + "calls": 19 + }, + "add_sub": { + "seconds": 0.0055407170000000006, + "calls": 23 + }, + "wire_import": { + "seconds": 0.001330797, + "calls": 6 + }, + "wire_export": { + "seconds": 0.00015322300000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.015019135000000001, + "calls": 1 + } + }, + "residue": { + "forward_ntt": { + "seconds": 0.035708193, + "calls": 418 + }, + "inverse_ntt": { + "seconds": 0.014244742000000001, + "calls": 180 + }, + "to_residues": { + "seconds": 0.004256384, + "calls": 10 + }, + "crt_reconstruct": { + "seconds": 0.0058745360000000005, + "calls": 8 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.0022139670000000003, + "calls": 399 + }, + "pointwise": { + "seconds": 0.008133528000000001, + "calls": 35 + }, + "scale_round": { + "seconds": 0.006246996, + "calls": 7 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.00052473, + "calls": 27 + }, + "automorphism": { + "seconds": 0.000875448, + "calls": 19 + }, + "add_sub": { + "seconds": 0.001730596, + "calls": 42 + }, + "wire_import": { + "seconds": 0.001332047, + "calls": 6 + }, + "wire_export": { + "seconds": 0.000152533, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.0018317960000000001, + "calls": 23 + }, + "other": { + "seconds": 0.0030155610000000004, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 737480, + "public_keys_bytes": 85602527, + "query_bytes": 368754, + "response_bytes": 102494, + "query_plus_response_bytes": 471248 + }, + "caches": { + "native": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 0, + "switch_keys": 18, + "key_payload_bytes": 56623104, + "server_plan_bytes": 655360 + }, + "residue": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 31, + "distance": 0 + }, + { + "index": 19, + "distance": 241 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "846b003afc224e40d2c5edea25ffd76fc2235c343fb6f37dcfe5dffef4a56219", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "499ed412000271c75c1980a4f2f312ef62d5da069f7f0995957a0de77dd2ea0a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "0d8b6a02fddf6b58996da91146ad161ee88783475d87edefe93946ae3defb0c6", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "f76e007558fbd1f6dd6ed6929770cddc6573568fc1bec9af2b6eca31afbd24e8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "1f468bf78fba83753839eddda17b1709f05abfb9b75b4950b7a782065268b15f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "48c2b2c7c8fcc8c957009cf328963026b15baf03f765a35cb616ffe02e875c00", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "2b6e18633fd8e07e7ec5527b7d4ccba2de5a4aa866453e4c32a4e48bc731c8a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "a2fe4ca8273a3f051ba44b0a1614bc19ce757939eeb37fc57835285392682356", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "99b513895a7baca7eebb65f5a1cb3a7d10b297b14479a327269828aa1f91058f", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "fa0315210c10b41cae83fad5fb4bf493354d76837cd898a7410d32ca623e92a5", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "4ff544e8e7371aa0800c7076d1fde9f9ba4bc20143444596618be5ae1cedad47", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "d9505ac8de6f7c356512111eee9e3d8ae19dd046034a81658b021db4596ecea8", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_residue_prime_control_1024.json b/benchmarks/results/native_bfv_residue_prime_control_1024.json new file mode 100644 index 0000000..2bd3110 --- /dev/null +++ b/benchmarks/results/native_bfv_residue_prime_control_1024.json @@ -0,0 +1,221 @@ +{ + "environment": { + "utc": "2026-09-11T02:23:26.903404+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "28170049a93a15b1a06a42f5ae89d74e80f50be2", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--backends", + "native", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_residue_prime_control_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "fffffffffffffffffffffffffffffffffffffffffffd1", + "ciphertext_modulus_primes": null, + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.5147744179703295, + "index_encrypt_s": 7.154086224967614, + "query_encrypt_s": 0.10949854203499854, + "public_key_export_s": 0.8259541729930788, + "native_public_key_import_s": 0.8619675009977072 + }, + "runs": [ + { + "backend": "native", + "repeat": 0, + "server_compute_s": 5.855862850090489 + }, + { + "backend": "native", + "repeat": 1, + "server_compute_s": 5.464001099811867 + }, + { + "backend": "native", + "repeat": 2, + "server_compute_s": 5.455950492061675 + }, + { + "backend": "native", + "repeat": 3, + "server_compute_s": 5.4547371591906995 + } + ], + "summary": { + "native": { + "first_search_s": 5.855862850090489, + "warm_searches_s": [ + 5.464001099811867, + 5.455950492061675, + 5.4547371591906995 + ], + "warm_median_s": 5.455950492061675 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "native": { + "first": 1.0, + "warm_median": 1.0 + } + }, + "seal_comparison": null, + "native_phases": { + "native": { + "forward_ntt": { + "seconds": 1.5736347460000002, + "calls": 18280 + }, + "inverse_ntt": { + "seconds": 0.5910954350000001, + "calls": 7480 + }, + "to_residues": { + "seconds": 0.132091116, + "calls": 256 + }, + "crt_reconstruct": { + "seconds": 0.854933025, + "calls": 1726 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.062764579, + "calls": 703 + }, + "pointwise": { + "seconds": 0.446116137, + "calls": 895 + }, + "scale_round": { + "seconds": 0.16660401900000002, + "calls": 193 + }, + "mod_q": { + "seconds": 0.504614024, + "calls": 1534 + }, + "buffers": { + "seconds": 0.022925539, + "calls": 1023 + }, + "automorphism": { + "seconds": 0.36388088900000004, + "calls": 639 + }, + "add_sub": { + "seconds": 0.18627094300000002, + "calls": 767 + }, + "wire_import": { + "seconds": 0.029229452000000003, + "calls": 130 + }, + "wire_export": { + "seconds": 0.00016510300000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.5101353780000001, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 23598495, + "public_keys_bytes": 85602362, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "native": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 0, + "switch_keys": 18, + "key_payload_bytes": 56623104, + "server_plan_bytes": 655360 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "846b003afc224e40d2c5edea25ffd76fc2235c343fb6f37dcfe5dffef4a56219", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "499ed412000271c75c1980a4f2f312ef62d5da069f7f0995957a0de77dd2ea0a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "0d8b6a02fddf6b58996da91146ad161ee88783475d87edefe93946ae3defb0c6", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "f76e007558fbd1f6dd6ed6929770cddc6573568fc1bec9af2b6eca31afbd24e8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "1f468bf78fba83753839eddda17b1709f05abfb9b75b4950b7a782065268b15f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "48c2b2c7c8fcc8c957009cf328963026b15baf03f765a35cb616ffe02e875c00", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "2b6e18633fd8e07e7ec5527b7d4ccba2de5a4aa866453e4c32a4e48bc731c8a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "a2fe4ca8273a3f051ba44b0a1614bc19ce757939eeb37fc57835285392682356", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "99b513895a7baca7eebb65f5a1cb3a7d10b297b14479a327269828aa1f91058f", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "fa0315210c10b41cae83fad5fb4bf493354d76837cd898a7410d32ca623e92a5", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "4ff544e8e7371aa0800c7076d1fde9f9ba4bc20143444596618be5ae1cedad47", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "d9505ac8de6f7c356512111eee9e3d8ae19dd046034a81658b021db4596ecea8" + } +} diff --git a/benchmarks/results/native_bfv_review_8192.json b/benchmarks/results/native_bfv_review_8192.json new file mode 100644 index 0000000..1368861 --- /dev/null +++ b/benchmarks/results/native_bfv_review_8192.json @@ -0,0 +1,145 @@ +{ + "environment": { + "utc": "2026-09-17T19:15:01.868692+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "be71b0dd4fa3693f6a6628e61f31e4b54ece9760", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_client_matrix.py", + "--num-vectors", + "8192", + "--embed-len", + "512", + "--poly-modulus-degree", + "16384", + "--server-backend", + "residue", + "--rns-modulus", + "--max-public-key-chars", + "268435456", + "--variants", + "bfv-packed", + "--repeats", + "1", + "--json-out", + "benchmarks/results/native_bfv_review_8192.json" + ], + "measurement_notes": "CPU synthetic experiment; no equivalent-security claim. Keys are public JSON including BFV evaluation keys. Index/query/response use common MessagePack framing with LE integer bytes and implicit IDs. No network/HTTP/TLS, secret-key, plaintext-content or AES-content bytes. Timings are separate from serialization; no concurrent test/benchmark processes intended.", + "seed": 1337, + "runs": [ + { + "variant": "bfv-packed", + "server_backend": "residue", + "vectors": 8192, + "embed_len": 512, + "config": { + "embed_len": 512, + "poly_modulus_degree": 16384, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "compact": true, + "timings": { + "key_generation_s": 9.108531635720283, + "public_key_export_s": 1.587051787879318, + "public_key_import_s": 2.978017693851143, + "index_encrypt_s": 56.27537882234901, + "query_encrypt_s": 0.2093489640392363, + "index_serialize_s": 0.4128873459994793, + "query_serialize_s": 0.0010858788155019283, + "index_deserialize_s": 0.2603874821215868, + "query_deserialize_s": 0.00039217714220285416, + "server_compute_s": 20.878492071758956, + "response_serialize_s": 0.00013659195974469185, + "response_deserialize_s": 0.00012690294533967972, + "client_decode_s": 0.0861029070802033 + }, + "sizes": { + "encrypted_index_bytes": 188765728, + "public_keys_bytes": 171204881, + "query_bytes": 737394, + "response_bytes": 204900, + "query_plus_response_bytes": 942294 + }, + "index_ciphertexts": 256, + "response_ciphertexts": 1, + "minimum_response_noise_budget_bits": 25, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 8191, + "distance": 0 + }, + { + "index": 2654, + "distance": 217 + } + ], + "correct": true, + "repeat": 0 + } + ], + "source_sha256": { + "benchmarks/bfv_client_matrix.py": "2f93923f21f3b9165ddc510ce029d61e8df873c7b519c1d61a99a193697a2f38", + "src/xtrace_sdk/x_vec/crypto/bfv_attested_client.py": "d3bed7a596fd19f834ea308fc332f0a28109a1136c7dd63026819b4cc106c7de", + "src/xtrace_sdk/x_vec/crypto/paillier_client.py": "40582d7acb86ee7dd2060e2d795c76829f4879db3b1eca9518be7a9599de4788", + "src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py": "2e684a8a7f2364a1134aa0271ee34fc6e2562367f26ae453f61a03fa80e31db3", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "6a384b16c347d84f90e40ccc9903b1ae916ae277e6171bd668181366d56e88a0", + "src/xtrace_sdk/x_vec/crypto/goldwasser_micali_client.py": "0982ad7532d07c8e26a701359ce8f4f7d4c1a3beb714050e741f3b534d59935f", + "src/xtrace_sdk/x_vec/crypto/paillier_lookup_client.py": "4faa0dd71d03b3e9bc9e49c4e60e8be7b173ff514b52cf1d94a3b1a2426bf699", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier_lookup.py": "eed0be7e0e2c1f6e8e8c0138262ec071fef73648e334a948ef1f82df6e9b9712", + "src/xtrace_sdk/x_vec/crypto/encryption/__init__.py": "261a50ed1758cee38c880cc625187f8acb051c6d5c96ba7a71ee956bdbea2cc5", + "src/xtrace_sdk/x_vec/crypto/encryption/goldwasser_micali.py": "4b32296c6210663c098718fa49bd810c427b1f8102d026d12a7d734f58e8e3d1", + "src/xtrace_sdk/x_vec/crypto/encryption/homomorphic_base.py": "915da9e500d0a142651243bb13de7631dc96103fea2ec9e5b9288bee498010e7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/paillier.py": "525a1944caef38d5d216ae84fcac982627d2a2fddea50b747e47ef05e2e91c9f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_private.py": "b1291e6d017be84638b054fb84e651734a44b2d54d601eb50dae2e9d2016bdfa", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/encryption/aes.py": "2c833351db3b249c381e315307d858791ee4e5b633d3465127cad763b4d095d5", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_decoder.h": "66aceec7cc8452dabdde0061d347195e49a9500fba7bcbbe805c967b7171d767", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp": "7b48d92e526147d97d9401e868c8d3e9dbf0b8ba567a67e05047e01f2a371fc0", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_private.cpython-312-x86_64-linux-gnu.so": "c55be18d74e56a1d6bb82c9fa31f55cb9ecf568aecf191a26278a660502b6c72", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "ee3f7371f92ba9fa3ebb8037906fc23c5f7832f96908d5879f1a4326847ffb53", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + }, + "median_timings": { + "bfv-packed": { + "key_generation_s": 9.108531635720283, + "public_key_export_s": 1.587051787879318, + "public_key_import_s": 2.978017693851143, + "index_encrypt_s": 56.27537882234901, + "query_encrypt_s": 0.2093489640392363, + "index_serialize_s": 0.4128873459994793, + "query_serialize_s": 0.0010858788155019283, + "index_deserialize_s": 0.2603874821215868, + "query_deserialize_s": 0.00039217714220285416, + "server_compute_s": 20.878492071758956, + "response_serialize_s": 0.00013659195974469185, + "response_deserialize_s": 0.00012690294533967972, + "client_decode_s": 0.0861029070802033 + } + } +} diff --git a/benchmarks/results/native_bfv_review_parameter_estimates.json b/benchmarks/results/native_bfv_review_parameter_estimates.json new file mode 100644 index 0000000..e64d2d5 --- /dev/null +++ b/benchmarks/results/native_bfv_review_parameter_estimates.json @@ -0,0 +1,521 @@ +{ + "utc": "2026-09-11T13:05:22.031193+00:00", + "sage": "10.9", + "python": "3.14.6", + "estimator_revision": "53da5982597709ba0fdf94ea37a84d822310fd84", + "runner_sha256": "8161c495e2b6932a5321b19dfa4da56f25b09934ae6df0b98a36d53e64425b7b", + "manifest": { + "schema": 1, + "profile": "review", + "config": { + "embed_len": 512, + "poly_modulus_degree": 16384, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "q": "1532495540865191964058577030327937992314109603578642433", + "q_primes": [ + 1152921504606748673, + 1152921504606683137, + 1152921504606584833 + ], + "secret": { + "distribution": "uniform", + "inclusive_bounds": [ + -1, + 1 + ] + }, + "error": { + "distribution": "centered_binomial", + "eta": 21, + "variance": 10.5 + }, + "ephemeral_u": { + "distribution": "uniform", + "inclusive_bounds": [ + -1, + 1 + ] + }, + "evaluation_keys": { + "galois_exponents": [ + 385, + 1537, + 3073, + 6145, + 8193, + 12289, + 14529, + 15617, + 16001, + 16385, + 17153, + 20481, + 22337, + 24577, + 26625, + 29697, + 31233 + ], + "digits_per_key": 6, + "key_switch_pairs": 108, + "note": "Encryptions of gadget-scaled s^2 and automorphic secrets. Ordinary LWE estimates do not justify these circular/key-dependent-message assumptions." + }, + "sdk_revision": "0cd7b41454533ab27082667cb9cb2f3c8040e823", + "worst_supported_hamming_bound": { + "encrypted": 688149, + "difference": 1427785, + "squared": 52059769849576678535, + "key_switch": 2216615439532032, + "tile": 28621377361628378284959515398718, + "merged": 14654145209153730814589761537631682, + "compacted": 73729, + "original_rounding_error_numerator": 960388714572308056395769201895141795266, + "terminal_rounding_error_numerator": 9124978689, + "original_modulus": 1532495540865191964058577030327937992314109603578642433, + "terminal_modulus": 1125899906842597, + "sufficient": true + }, + "source_sha256": { + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/bfv_assurance.py": "ba64de6db3f76031ba9c35627a0a3790244c816937a97a3b65c98b9c699bb33d", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } + }, + "command": [ + "benchmarks/bfv_parameter_audit.py", + "estimate", + "--manifest", + "/tmp/xtrace-bfv-review-manifest.json", + "--estimator", + "/tmp/xtrace-bfv-lattice-estimator", + "--json-out", + "benchmarks/results/native_bfv_review_parameter_estimates.json" + ], + "interpretation": "Heuristic generic LWE attack estimates with Gaussian-shape approximation for centered binomial errors. Reports m=N and unbounded independent samples; RLWE structure, evaluation-key assumptions, decryption failures, implementation leakage and protocol attacks are not certified. ADPS16-quantum is a quantum sieving cost sensitivity model, not a complete quantum security analysis. Unlisted attacks are not evaluated.", + "runs": [ + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "primal_usvp", + "seconds": 0.05196200706996024, + "log2_rop": 341.1723757172288, + "cost": { + "rop": "5.04799345125361e102", + "red": "5.04799345125361e102", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "primal_bdd", + "seconds": 5.101183895953, + "log2_rop": 340.7298617250473, + "cost": { + "rop": "3.71457214481230e102", + "red": "3.45038992834586e102", + "svp": "2.64182216466433e101", + "beta": "1090", + "eta": "1129", + "d": "32344", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "dual", + "seconds": 0.04392242501489818, + "log2_rop": 343.3327532463879, + "cost": { + "rop": "2.25661370326375e103", + "mem": "27004289658070035888654106211172915795011311755896265023025172176204854", + "m": "16204", + "beta": "1096", + "d": "32588", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "MATZOV-classical", + "attack": "dual_hybrid", + "seconds": 14.51800377201289, + "log2_rop": 339.58676911414483, + "cost": { + "rop": "1.68191341284720e102", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)", + "red": "1.68190843684125e102", + "guess": "4.97600594903040e96", + "beta": "1083", + "p": "3", + "zeta": "0", + "t": "190", + "beta_": "1066", + "N": "7.82287607371639e65", + "m": "16384" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "primal_usvp", + "seconds": 0.04052111296914518, + "log2_rop": 318.864, + "cost": { + "rop": "9.71915565210739e95", + "red": "9.71915565210739e95", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "primal_bdd", + "seconds": 5.235912185860798, + "log2_rop": 319.160068553334, + "cost": { + "rop": "1.19331212308321e96", + "red": "1.18995159682775e96", + "svp": "3.36052625546777e93", + "beta": "1093", + "eta": "1063", + "d": "32383", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "dual", + "seconds": 0.03102129092440009, + "log2_rop": 319.73999999999995, + "cost": { + "rop": "1.78373696975310e96", + "mem": "273648654219124481565040050707290299685135857117579618854340086718143754", + "m": "16193", + "beta": "1095", + "d": "32577", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "ADPS16-classical", + "attack": "dual_hybrid", + "seconds": 8.881262043956667, + "log2_rop": 316.82, + "cost": { + "rop": "2.35680269580672e95", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)", + "red": "2.35680269580672e95", + "guess": "3.23123409447432e77", + "beta": "1085", + "p": "3", + "zeta": "0", + "t": "150", + "beta_": "1085", + "N": "3.83009980795253e67", + "m": "16384" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "primal_usvp", + "seconds": 0.03971595619805157, + "log2_rop": 289.38, + "cost": { + "rop": "1.29437508644779e87", + "red": "1.29437508644779e87", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "primal_bdd", + "seconds": 5.272714625112712, + "log2_rop": 289.65199368299255, + "cost": { + "rop": "1.56292595152198e87", + "red": "1.55536776548515e87", + "svp": "7.55818603682939e84", + "beta": "1093", + "eta": "1063", + "d": "32383", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "dual", + "seconds": 0.02500018198043108, + "log2_rop": 290.175, + "cost": { + "rop": "2.24584092635855e87", + "mem": "273648654219124481565040050707290299685135857117579618854340086718143754", + "m": "16193", + "beta": "1095", + "d": "32577", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "N", + "model": "ADPS16-quantum", + "attack": "dual_hybrid", + "seconds": 8.84267289401032, + "log2_rop": 287.5250000013029, + "cost": { + "rop": "3.57807552817983e86", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=16384, tag=None)", + "red": "3.57807552494859e86", + "guess": "3.23123409447432e77", + "beta": "1085", + "p": "3", + "zeta": "0", + "t": "150", + "beta_": "1085", + "N": "3.83009980795253e67", + "m": "16384" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "primal_usvp", + "seconds": 0.04323683097027242, + "log2_rop": 341.1723757172288, + "cost": { + "rop": "5.04799345125361e102", + "red": "5.04799345125361e102", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "primal_bdd", + "seconds": 5.31509437202476, + "log2_rop": 340.7298617250473, + "cost": { + "rop": "3.71457214481230e102", + "red": "3.45038992834586e102", + "svp": "2.64182216466433e101", + "beta": "1090", + "eta": "1129", + "d": "32344", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "dual", + "seconds": 0.03988387901335955, + "log2_rop": 343.3327532463879, + "cost": { + "rop": "2.25661370326375e103", + "mem": "27004289658070035888654106211172915795011311755896265023025172176204854", + "m": "16204", + "beta": "1096", + "d": "32588", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "MATZOV-classical", + "attack": "dual_hybrid", + "seconds": 14.420737589942291, + "log2_rop": 339.58676911414483, + "cost": { + "rop": "1.68191341284720e102", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "1.68190843684125e102", + "guess": "4.97600594903040e96", + "beta": "1083", + "p": "3", + "zeta": "0", + "t": "190", + "beta_": "1066", + "N": "7.82287607371639e65", + "m": "16384" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "primal_usvp", + "seconds": 0.040466082049533725, + "log2_rop": 318.864, + "cost": { + "rop": "9.71915565210739e95", + "red": "9.71915565210739e95", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "primal_bdd", + "seconds": 5.354171877028421, + "log2_rop": 319.160068553334, + "cost": { + "rop": "1.19331212308321e96", + "red": "1.18995159682775e96", + "svp": "3.36052625546777e93", + "beta": "1093", + "eta": "1063", + "d": "32383", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "dual", + "seconds": 0.027775981929153204, + "log2_rop": 319.73999999999995, + "cost": { + "rop": "1.78373696975310e96", + "mem": "273648654219124481565040050707290299685135857117579618854340086718143754", + "m": "16193", + "beta": "1095", + "d": "32577", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-classical", + "attack": "dual_hybrid", + "seconds": 8.707200119970366, + "log2_rop": 316.82, + "cost": { + "rop": "2.35680269580672e95", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "2.35680269580672e95", + "guess": "3.23123409447432e77", + "beta": "1085", + "p": "3", + "zeta": "0", + "t": "150", + "beta_": "1085", + "N": "3.83009980795253e67", + "m": "16384" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "primal_usvp", + "seconds": 0.040943962056189775, + "log2_rop": 289.38, + "cost": { + "rop": "1.29437508644779e87", + "red": "1.29437508644779e87", + "delta": "1.00191078002672", + "beta": "1092", + "d": "32083", + "tag": "usvp", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "primal_bdd", + "seconds": 5.37198440101929, + "log2_rop": 289.65199368299255, + "cost": { + "rop": "1.56292595152198e87", + "red": "1.55536776548515e87", + "svp": "7.55818603682939e84", + "beta": "1093", + "eta": "1063", + "d": "32383", + "tag": "bdd", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "dual", + "seconds": 0.026228047907352448, + "log2_rop": 290.175, + "cost": { + "rop": "2.24584092635855e87", + "mem": "273648654219124481565040050707290299685135857117579618854340086718143754", + "m": "16193", + "beta": "1095", + "d": "32577", + "repetitions": "1", + "tag": "dual" + } + }, + { + "samples": "unbounded", + "model": "ADPS16-quantum", + "attack": "dual_hybrid", + "seconds": 8.744628780055791, + "log2_rop": 287.5250000013029, + "cost": { + "rop": "3.57807552817983e86", + "problem": "LWEParameters(n=16384, q=1532495540865191964058577030327937992314109603578642433, Xs=D(\u03c3=0.82), Xe=D(\u03c3=3.24), m=+Infinity, tag=None)", + "red": "3.57807552494859e86", + "guess": "3.23123409447432e77", + "beta": "1085", + "p": "3", + "zeta": "0", + "t": "150", + "beta_": "1085", + "N": "3.83009980795253e67", + "m": "16384" + } + } + ], + "complete": true +} diff --git a/benchmarks/results/native_bfv_rns_1024.json b/benchmarks/results/native_bfv_rns_1024.json new file mode 100644 index 0000000..c3b34eb --- /dev/null +++ b/benchmarks/results/native_bfv_rns_1024.json @@ -0,0 +1,222 @@ +{ + "environment": { + "utc": "2026-09-10T23:15:20.306784+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "ff8d5a29a58874e6498e1cdae6b3fcde21e19a5d", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--backends", + "optimized,rns", + "--seal", + "--repeats", + "3", + "--json-out", + "benchmarks/results/native_bfv_rns_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for both backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.33735747798346, + "index_encrypt_s": 6.699632134055719, + "query_encrypt_s": 0.1024328120984137, + "public_key_export_s": 0.7893146618735045, + "optimized_public_key_import_s": 0.8305047010071576, + "rns_public_key_import_s": 0.8661916260607541 + }, + "runs": [ + { + "backend": "optimized", + "repeat": 0, + "server_compute_s": 55.7612584088929 + }, + { + "backend": "rns", + "repeat": 0, + "server_compute_s": 8.329127850010991 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 2.4031696289312094 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 2.387246180092916 + }, + { + "backend": "rns", + "repeat": 1, + "server_compute_s": 8.029489122098312 + }, + { + "backend": "optimized", + "repeat": 1, + "server_compute_s": 55.893618842121214 + }, + { + "backend": "optimized", + "repeat": 2, + "server_compute_s": 55.91135185002349 + }, + { + "backend": "rns", + "repeat": 2, + "server_compute_s": 8.04160486208275 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 2.3929480500519276 + } + ], + "summary": { + "optimized": { + "first_search_s": 55.7612584088929, + "warm_searches_s": [ + 55.893618842121214, + 55.91135185002349 + ], + "warm_median_s": 55.90248534607235 + }, + "rns": { + "first_search_s": 8.329127850010991, + "warm_searches_s": [ + 8.029489122098312, + 8.04160486208275 + ], + "warm_median_s": 8.03554699209053 + }, + "seal": { + "first_search_s": 2.4031696289312094, + "warm_searches_s": [ + 2.387246180092916, + 2.3929480500519276 + ], + "warm_median_s": 2.3900971150724217 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "optimized": { + "first": 1.0, + "warm_median": 1.0 + }, + "rns": { + "first": 6.694729557887542, + "warm_median": 6.956898565971642 + }, + "seal": { + "first": 23.20321367979849, + "warm_median": 23.389210837308788 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 27680661, + "query_bytes": 432556, + "public_key_bytes": 39587127, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 105915, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "sizes": { + "encrypted_index_bytes": 23598495, + "public_keys_bytes": 85603545, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "optimized": { + "switch_keys": 16, + "key_payload_bytes": 52083728, + "transform_table_bytes": 0 + }, + "rns": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "switch_keys": 16, + "key_payload_bytes": 50331648 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "839279a7bfd41e46e02a4f49f51afc6e3fbe2e3c33d5a4f9745eb3107bf399c1", + "benchmarks/bfv_client_matrix.py": "b5539dd816f6dbd8f26fadde4f72c9582a8fa4f67bf8a64b3a874d38f576216c", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "03205e7775265873da6da51a2d26aa0e63a59c2961dab31f29e71e8b9b92a84f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "de206a348b704982cfa58526189af7c61cf0628807457f7beddb00498a2ea78a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "6f97a52f5b4b9981533fbed5d043e4e2179906491d0805609fd76e025e38bbc9", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "fb72e876eb585874c8dd42a6ecba89ed8f85b1b509f18171cfa15119384ce801", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "13f4e8aab63ce7cbab5326cab67676ee3c2a9c3cbb9786baa18bed9833c4a90e", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "3263d9f575c947aaa3a9199b542dac7e607afdfbb223d70449a2f173be3df78c", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "6fc7f25165300eda1ecd5bd9d516ba958f045de8951d81ff296c72ce461b5974", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_rns_32.json b/benchmarks/results/native_bfv_rns_32.json new file mode 100644 index 0000000..f6b06ca --- /dev/null +++ b/benchmarks/results/native_bfv_rns_32.json @@ -0,0 +1,242 @@ +{ + "environment": { + "utc": "2026-09-10T23:06:32.636539+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "ff8d5a29a58874e6498e1cdae6b3fcde21e19a5d", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "32", + "--backends", + "optimized,rns", + "--seal", + "--repeats", + "4", + "--profile-dir", + "/tmp/xtrace-bfv-rns-profiles", + "--json-out", + "benchmarks/results/native_bfv_rns_32.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for both backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 32, + "setup_timings": { + "key_generation_s": 4.319804938975722, + "index_encrypt_s": 0.21808502497151494, + "query_encrypt_s": 0.1036710909102112, + "public_key_export_s": 0.7804761249572039, + "optimized_public_key_import_s": 0.8426786060445011, + "rns_public_key_import_s": 0.8472385408822447 + }, + "runs": [ + { + "backend": "optimized", + "repeat": 0, + "server_compute_s": 1.7459196471609175 + }, + { + "backend": "rns", + "repeat": 0, + "server_compute_s": 0.5007300598081201 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 0.07752445782534778 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 0.07478765211999416 + }, + { + "backend": "rns", + "repeat": 1, + "server_compute_s": 0.26233402313664556 + }, + { + "backend": "optimized", + "repeat": 1, + "server_compute_s": 1.6846589841879904 + }, + { + "backend": "optimized", + "repeat": 2, + "server_compute_s": 1.6894245909061283 + }, + { + "backend": "rns", + "repeat": 2, + "server_compute_s": 0.2628466538153589 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 0.07481790310703218 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 0.07475750101730227 + }, + { + "backend": "rns", + "repeat": 3, + "server_compute_s": 0.2623735030647367 + }, + { + "backend": "optimized", + "repeat": 3, + "server_compute_s": 1.6904642400331795 + } + ], + "summary": { + "optimized": { + "first_search_s": 1.7459196471609175, + "warm_searches_s": [ + 1.6846589841879904, + 1.6894245909061283, + 1.6904642400331795 + ], + "warm_median_s": 1.6894245909061283 + }, + "rns": { + "first_search_s": 0.5007300598081201, + "warm_searches_s": [ + 0.26233402313664556, + 0.2628466538153589, + 0.2623735030647367 + ], + "warm_median_s": 0.2623735030647367 + }, + "seal": { + "first_search_s": 0.07752445782534778, + "warm_searches_s": [ + 0.07478765211999416, + 0.07481790310703218, + 0.07475750101730227 + ], + "warm_median_s": 0.07478765211999416 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "optimized": { + "first": 1.0, + "warm_median": 1.0 + }, + "rns": { + "first": 3.4867482248418527, + "warm_median": 6.43900611598454 + }, + "seal": { + "first": 22.520888196267567, + "warm_median": 22.5896193157061 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 865100, + "query_bytes": 432689, + "public_key_bytes": 39584869, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 103240, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "sizes": { + "encrypted_index_bytes": 737480, + "public_keys_bytes": 85602442, + "query_bytes": 368754, + "response_bytes": 102494, + "query_plus_response_bytes": 471248 + }, + "caches": { + "optimized": { + "switch_keys": 11, + "key_payload_bytes": 35807568, + "transform_table_bytes": 0 + }, + "rns": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 4, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "switch_keys": 11, + "key_payload_bytes": 34603008 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 31, + "distance": 0 + }, + { + "index": 19, + "distance": 241 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "839279a7bfd41e46e02a4f49f51afc6e3fbe2e3c33d5a4f9745eb3107bf399c1", + "benchmarks/bfv_client_matrix.py": "b5539dd816f6dbd8f26fadde4f72c9582a8fa4f67bf8a64b3a874d38f576216c", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "03205e7775265873da6da51a2d26aa0e63a59c2961dab31f29e71e8b9b92a84f", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "de206a348b704982cfa58526189af7c61cf0628807457f7beddb00498a2ea78a", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "6f97a52f5b4b9981533fbed5d043e4e2179906491d0805609fd76e025e38bbc9", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "fb72e876eb585874c8dd42a6ecba89ed8f85b1b509f18171cfa15119384ce801", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "13f4e8aab63ce7cbab5326cab67676ee3c2a9c3cbb9786baa18bed9833c4a90e", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "3263d9f575c947aaa3a9199b542dac7e607afdfbb223d70449a2f173be3df78c", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "6fc7f25165300eda1ecd5bd9d516ba958f045de8951d81ff296c72ce461b5974", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_rns_profile_32.txt b/benchmarks/results/native_bfv_rns_profile_32.txt new file mode 100644 index 0000000..742185d --- /dev/null +++ b/benchmarks/results/native_bfv_rns_profile_32.txt @@ -0,0 +1,83 @@ +Additional cProfile runs for 32 vectors x 512 bits. +These instrumented runs are excluded from benchmark timings. + +optimized +Thu Sep 10 19:06:32 2026 /tmp/xtrace-bfv-rns-profiles/optimized.prof + + 3581976 function calls in 2.024 seconds + + Ordered by: cumulative time + List reduced from 74 to 30 due to restriction <30> + + ncalls tottime percall cumtime percall filename:lineno(function) + 1 0.001 0.001 2.046 2.046 bfv_server.py:109(evaluate) + 1 0.002 0.002 2.044 2.044 bfv_client.py:273(encode_hamming_server_packed) + 2 0.001 0.000 1.860 0.930 bfv_client.py:238(_distance_tile) + 2 0.022 0.011 1.859 0.930 bfv_evaluator.py:242(hamming_tile) + 19 0.008 0.000 1.318 0.069 bfv_evaluator.py:269(rotate_rows) + 21 0.000 0.000 1.304 0.062 bfv_evaluator.py:146(_switch) + 21 1.185 0.056 1.304 0.062 bfv_evaluator.py:55(apply) + 158 0.059 0.000 0.352 0.002 {built-in method builtins.any} + 2 0.000 0.000 0.333 0.166 bfv_evaluator.py:237(xor) + 2 0.003 0.001 0.315 0.157 bfv_evaluator.py:221(multiply) + 11 0.000 0.000 0.201 0.018 bfv.py:289(validate_ciphertext) + 180246 0.063 0.000 0.190 0.000 bfv.py:300() + 2 0.003 0.001 0.172 0.086 bfv.py:447(multiply) + 63 0.000 0.000 0.170 0.003 bfv_evaluator.py:118(_validate) + 19 0.000 0.000 0.164 0.009 bfv_evaluator.py:158(add) + 2 0.002 0.001 0.139 0.070 bfv_evaluator.py:198(relinearize) + 196657 0.030 0.000 0.139 0.000 {built-in method builtins.isinstance} + 21 0.000 0.000 0.112 0.005 bfv_evaluator.py:139(_pair) + 196632 0.020 0.000 0.108 0.000 :117(__instancecheck__) + 10 0.049 0.005 0.107 0.011 bfv.py:39(_poly_product) + 2 0.000 0.000 0.102 0.051 bfv_evaluator.py:186(multiply_plain) + 2 0.000 0.000 0.102 0.051 bfv.py:398(multiply_plain) + 42 0.019 0.000 0.099 0.002 bfv_evaluator.py:63(finish) + 196632 0.038 0.000 0.088 0.000 {built-in method _abc._abc_instancecheck} + 1048704 0.088 0.000 0.088 0.000 bfv_evaluator.py:133() + 2 0.000 0.000 0.074 0.037 bfv.py:351(_pair) + 57 0.016 0.000 0.062 0.001 bfv_evaluator.py:164() + 344106 0.062 0.000 0.062 0.000 bfv_evaluator.py:69() + 3 0.001 0.000 0.059 0.020 bfv.py:566(ciphertext_from_ints) + 57 0.000 0.000 0.055 0.001 bfv_evaluator.py:289() + + +rns +Thu Sep 10 19:06:32 2026 /tmp/xtrace-bfv-rns-profiles/rns.prof + + 770665 function calls in 0.342 seconds + + Ordered by: cumulative time + List reduced from 62 to 30 due to restriction <30> + + ncalls tottime percall cumtime percall filename:lineno(function) + 1 0.001 0.001 0.342 0.342 bfv_server.py:109(evaluate) + 1 0.002 0.002 0.340 0.340 bfv_client.py:273(encode_hamming_server_packed) + 2 0.000 0.000 0.213 0.107 bfv_client.py:238(_distance_tile) + 2 0.000 0.000 0.213 0.107 bfv_evaluator.py:242(hamming_tile) + 2 0.000 0.000 0.184 0.092 bfv_rns.py:68(hamming_tile) + 2 0.178 0.089 0.178 0.089 {built-in method xtrace_sdk.x_vec.crypto.bfv_cpu_ext._bfv_rns.hamming_tile} + 32 0.011 0.000 0.121 0.004 {built-in method builtins.any} + 5 0.000 0.000 0.090 0.018 bfv.py:289(validate_ciphertext) + 81930 0.028 0.000 0.085 0.000 bfv.py:300() + 98353 0.015 0.000 0.069 0.000 {built-in method builtins.isinstance} + 3 0.001 0.000 0.059 0.020 bfv.py:566(ciphertext_from_ints) + 98328 0.010 0.000 0.054 0.000 :117(__instancecheck__) + 98328 0.019 0.000 0.044 0.000 {built-in method _abc._abc_instancecheck} + 1 0.000 0.000 0.027 0.027 bfv.py:504(modulus_switch) + 98328 0.009 0.000 0.025 0.000 :121(__subclasscheck__) + 7 0.000 0.000 0.019 0.003 bfv_evaluator.py:118(_validate) + 2 0.001 0.000 0.018 0.009 bfv.py:244(_plaintext) + 1 0.000 0.000 0.017 0.017 bfv.py:545(ciphertext_to_ints) + 3 0.000 0.000 0.016 0.005 bfv_evaluator.py:139(_pair) + 98328 0.016 0.000 0.016 0.000 {built-in method _abc._abc_subclasscheck} + 16386 0.003 0.000 0.015 0.000 bfv.py:248() + 1 0.000 0.000 0.013 0.013 bfv_evaluator.py:269(rotate_rows) + 1 0.000 0.000 0.011 0.011 bfv_rns.py:59(rotate) + 114702 0.010 0.000 0.010 0.000 bfv_evaluator.py:133() + 1 0.000 0.000 0.009 0.009 bfv_evaluator.py:158(add) + 1 0.009 0.009 0.009 0.009 {built-in method xtrace_sdk.x_vec.crypto.bfv_cpu_ext._bfv_rns.rotate_rows} + 3 0.001 0.000 0.009 0.003 bfv.py:524() + 16386 0.004 0.000 0.008 0.000 bfv.py:525() + 12 0.006 0.000 0.006 0.000 {built-in method gmpy2.gmpy2.unpack} + 6 0.000 0.000 0.004 0.001 bfv_rns.py:40(_unpack) diff --git a/benchmarks/results/native_bfv_security_1024.json b/benchmarks/results/native_bfv_security_1024.json new file mode 100644 index 0000000..5ceeb01 --- /dev/null +++ b/benchmarks/results/native_bfv_security_1024.json @@ -0,0 +1,432 @@ +{ + "environment": { + "utc": "2026-09-11T03:20:46.201242+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "06d8d172c168f37dd77d1572f68df2088d2d82d7", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--rns-modulus", + "--backends", + "residue", + "--residue-levels", + "0,2", + "--seal", + "--repeats", + "4", + "--native-profile", + "--json-out", + "benchmarks/results/native_bfv_security_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for all native backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, arithmetic context creation (reported under setup), key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "backend_variants": { + "residue-0": [ + "residue", + 0 + ], + "residue-2": [ + "residue", + 2 + ] + }, + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "ciphertext_modulus_hex": "ffffffffffbc0030000004bff78002fff1004bffbc001", + "ciphertext_modulus_primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137 + ], + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.603943218942732, + "index_encrypt_s": 6.865148832090199, + "query_encrypt_s": 0.10382973193190992, + "public_key_export_s": 0.8147107770200819, + "residue-0_public_key_import_s": 1.466961293015629, + "residue-0_arithmetic_context_s": 0.0017840259242802858, + "residue-2_public_key_import_s": 1.4780206978321075, + "residue-2_arithmetic_context_s": 0.0010454009752720594 + }, + "runs": [ + { + "backend": "residue-0", + "repeat": 0, + "server_compute_s": 3.0709580299444497 + }, + { + "backend": "residue-2", + "repeat": 0, + "server_compute_s": 2.7449227031320333 + }, + { + "backend": "seal", + "repeat": 0, + "server_compute_s": 2.484233553055674 + }, + { + "backend": "seal", + "repeat": 1, + "server_compute_s": 2.4849505240563303 + }, + { + "backend": "residue-2", + "repeat": 1, + "server_compute_s": 2.4235800479073077 + }, + { + "backend": "residue-0", + "repeat": 1, + "server_compute_s": 2.7881676598917693 + }, + { + "backend": "residue-0", + "repeat": 2, + "server_compute_s": 2.787437440128997 + }, + { + "backend": "residue-2", + "repeat": 2, + "server_compute_s": 2.426682349992916 + }, + { + "backend": "seal", + "repeat": 2, + "server_compute_s": 2.4957599870394915 + }, + { + "backend": "seal", + "repeat": 3, + "server_compute_s": 2.4768296871334314 + }, + { + "backend": "residue-2", + "repeat": 3, + "server_compute_s": 2.420860161073506 + }, + { + "backend": "residue-0", + "repeat": 3, + "server_compute_s": 2.7841470839921385 + } + ], + "summary": { + "residue-0": { + "first_search_s": 3.0709580299444497, + "warm_searches_s": [ + 2.7881676598917693, + 2.787437440128997, + 2.7841470839921385 + ], + "warm_median_s": 2.787437440128997 + }, + "residue-2": { + "first_search_s": 2.7449227031320333, + "warm_searches_s": [ + 2.4235800479073077, + 2.426682349992916, + 2.420860161073506 + ], + "warm_median_s": 2.4235800479073077 + }, + "seal": { + "first_search_s": 2.484233553055674, + "warm_searches_s": [ + 2.4849505240563303, + 2.4957599870394915, + 2.4768296871334314 + ], + "warm_median_s": 2.4849505240563303 + } + }, + "first_search_speedup": null, + "warm_median_speedup": null, + "speedups_relative_to_first_native_backend": { + "residue-0": { + "first": 1.0, + "warm_median": 1.0 + }, + "residue-2": { + "first": 1.1187775985241408, + "warm_median": 1.150132194946839 + }, + "seal": { + "first": 1.236179273952358, + "warm_median": 1.1217275407072893 + } + }, + "seal_comparison": { + "tenseal": "0.3.16", + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "active_coeff_modulus_bits": [ + 43, + 43, + 44, + 44 + ], + "key_coeff_modulus_bits": [ + 43, + 43, + 44, + 44, + 44 + ], + "security_setting": "SEAL TC128; no equivalent-security claim for native parameters", + "index_bytes": 27680519, + "query_bytes": 432613, + "public_key_bytes": 39585853, + "timing_notes": "SEAL search includes its serialization, MessagePack and binding temporary-file I/O. Native times exclude outer MessagePack; both include ciphertext decoding, arithmetic, compaction and ciphertext encoding. These are practical server calls with different parameters/formats, not identical arithmetic kernels.", + "response_bytes": 105854, + "all_distances_correct": true, + "minimum_response_noise_budget_bits": 19 + }, + "native_phases": { + "residue-0": { + "forward_ntt": { + "seconds": 1.1965777880000001, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.457320508, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.13917984900000002, + "calls": 320 + }, + "crt_reconstruct": { + "seconds": 0.18413337100000002, + "calls": 194 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.079418079, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.29637719700000004, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.16709886100000002, + "calls": 193 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.019559113, + "calls": 895 + }, + "automorphism": { + "seconds": 0.033134543, + "calls": 639 + }, + "add_sub": { + "seconds": 0.058943964, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.029082671, + "calls": 130 + }, + "wire_export": { + "seconds": 0.000167453, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.061222676000000004, + "calls": 705 + }, + "rns_scale": { + "seconds": 0.0, + "calls": 0 + }, + "other": { + "seconds": 0.07879342, + "calls": 1 + } + }, + "residue-2": { + "forward_ntt": { + "seconds": 1.1098617400000002, + "calls": 13934 + }, + "inverse_ntt": { + "seconds": 0.39522169, + "calls": 5946 + }, + "to_residues": { + "seconds": 0.08329718, + "calls": 128 + }, + "crt_reconstruct": { + "seconds": 0.00039382800000000004, + "calls": 2 + }, + "crt_exact_fallback": { + "seconds": 0.0, + "calls": 0 + }, + "gadget_decompose": { + "seconds": 0.080379433, + "calls": 13357 + }, + "pointwise": { + "seconds": 0.29532529300000004, + "calls": 1151 + }, + "scale_round": { + "seconds": 0.001434909, + "calls": 1 + }, + "mod_q": { + "seconds": 0.0, + "calls": 0 + }, + "buffers": { + "seconds": 0.018826629, + "calls": 895 + }, + "automorphism": { + "seconds": 0.039910895, + "calls": 639 + }, + "add_sub": { + "seconds": 0.058128008, + "calls": 1406 + }, + "wire_import": { + "seconds": 0.029310127000000002, + "calls": 130 + }, + "wire_export": { + "seconds": 0.00017654400000000002, + "calls": 2 + }, + "rns_compose": { + "seconds": 0.060768088000000005, + "calls": 705 + }, + "rns_scale": { + "seconds": 0.21904145000000003, + "calls": 192 + }, + "other": { + "seconds": 0.056853829, + "calls": 1 + } + } + }, + "native_profile_note": "Additional warm searches, excluded from performance samples. Exclusive wall times and call counts; other includes Python, binding overhead and unscoped native work. buffers covers explicit buffer setup; GMP allocations remain charged to their arithmetic phases. Profiles are per-thread and restored after exceptions.", + "sizes": { + "encrypted_index_bytes": 23598496, + "public_keys_bytes": 85604170, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "caches": { + "residue-0": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 2752512, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 0, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 589824 + }, + "residue-2": { + "primes": [ + 1152921504606830593, + 1152921504606748673, + 1152921504606683137, + 1152921504606601217, + 1152921504606584833, + 1152921504606109697, + 1152921504605962241 + ], + "switch_prime_count": 3, + "transform_table_bytes": 1835232, + "gmp": "6.3.0", + "compiler": "13.3.0", + "fast_arithmetic": true, + "residue_prime_count": 3, + "kernel_level": 2, + "switch_keys": 18, + "key_payload_bytes": 42467328, + "server_plan_bytes": 591648 + } + }, + "cache_size_note": "Additional to the existing public key: GMP object payloads including folding constants, or native NTT coefficient arrays and transform tables. Excludes containers, CRT constants and transient allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "ad70ee74af27d055ac9653a58ed46b3c58381aef2de875568cdd15eeb5a9fac6", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "05e30fc3b8191e2fd0f0620ac7dbbc35ccea0a43913fc62f66575b0d44716aeb", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "experiments/bfv/packed_hamming.py": "5c940efe69617a822d0e6fce8afff92d14804cd49431cc87705553b71a709e01" + } +} diff --git a/benchmarks/results/native_bfv_server_1024.json b/benchmarks/results/native_bfv_server_1024.json new file mode 100644 index 0000000..0a3c7ca --- /dev/null +++ b/benchmarks/results/native_bfv_server_1024.json @@ -0,0 +1,128 @@ +{ + "environment": { + "utc": "2026-09-10T22:25:48.950450+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "cf454e5eb6e053822501645bd44c6f23aa832a19", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "1024", + "--repeats", + "3", + "--json-out", + "benchmarks/results/native_bfv_server_1024.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for both backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 1024, + "setup_timings": { + "key_generation_s": 4.636117450892925, + "index_encrypt_s": 7.034667205996811, + "query_encrypt_s": 0.10684434906579554, + "public_key_export_s": 0.8335895461495966, + "reference_public_key_import_s": 0.850766752846539, + "optimized_public_key_import_s": 0.8723104181699455 + }, + "runs": [ + { + "backend": "reference", + "repeat": 0, + "server_compute_s": 121.56445405795239 + }, + { + "backend": "optimized", + "repeat": 0, + "server_compute_s": 60.35474809096195 + }, + { + "backend": "optimized", + "repeat": 1, + "server_compute_s": 60.588989215902984 + }, + { + "backend": "reference", + "repeat": 1, + "server_compute_s": 121.66320819617249 + }, + { + "backend": "reference", + "repeat": 2, + "server_compute_s": 114.82569716288708 + }, + { + "backend": "optimized", + "repeat": 2, + "server_compute_s": 56.66450631082989 + } + ], + "summary": { + "reference": { + "first_search_s": 121.56445405795239, + "warm_searches_s": [ + 121.66320819617249, + 114.82569716288708 + ], + "warm_median_s": 118.24445267952979 + }, + "optimized": { + "first_search_s": 60.35474809096195, + "warm_searches_s": [ + 60.588989215902984, + 56.66450631082989 + ], + "warm_median_s": 58.62674776336644 + } + }, + "first_search_speedup": 2.014165544602058, + "warm_median_speedup": 2.0169028163867573, + "sizes": { + "encrypted_index_bytes": 23598492, + "public_keys_bytes": 85603614, + "query_bytes": 368754, + "response_bytes": 102496, + "query_plus_response_bytes": 471250 + }, + "optimized_cached_switch_keys": 16, + "optimized_packed_cache_payload_bytes": 52083728, + "cache_size_note": "sys.getsizeof of packed GMP values, including folding constants; additional to the existing public key. Excludes small Python containers and transient arithmetic allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 1023, + "distance": 0 + }, + { + "index": 81, + "distance": 220 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "c0523cfbd4100657fc7f9eae9ac117d5fa86fc57e9983a7e2dd840920a2c2298", + "benchmarks/bfv_client_matrix.py": "d7c4c4242ad2cc537b8c39cf990944b83c3a2f178ce317e23dd415e5c12f7283", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "1a9927c6e15937ee39e6e84131890e249e14352795c21e8b8930cfe1007dd674", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "6e2c84e7882bb86f8e623c2eb2d104e0d4cab85090aed17840adc0809a5cd9a3", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3" + } +} diff --git a/benchmarks/results/native_bfv_server_32.json b/benchmarks/results/native_bfv_server_32.json new file mode 100644 index 0000000..207bd25 --- /dev/null +++ b/benchmarks/results/native_bfv_server_32.json @@ -0,0 +1,142 @@ +{ + "environment": { + "utc": "2026-09-10T22:15:47.796934+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "revision": "cf454e5eb6e053822501645bd44c6f23aa832a19", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_server.py", + "--num-vectors", + "32", + "--repeats", + "4", + "--profile-dir", + "/tmp/xtrace-bfv-server-profiles", + "--json-out", + "/tmp/xtrace-bfv-server-32.json" + ], + "measurement_notes": "One CPU process, identical public keys and encrypted inputs for both backends. Server times include native ciphertext decoding/encoding, terminal modulus switching and lazy cache preparation; exclude outer MessagePack, key import, encryption and client decryption. First search starts with empty key and mask caches. Warm searches reuse these caches. Order alternates. Profiles, if requested, are additional untimed searches. No network or equal-security comparison to other schemes.", + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 32, + "setup_timings": { + "key_generation_s": 4.557773574022576, + "index_encrypt_s": 0.21618583402596414, + "query_encrypt_s": 0.10164761217311025, + "public_key_export_s": 0.8099749418906868, + "reference_public_key_import_s": 0.8771210089325905, + "optimized_public_key_import_s": 0.8737351400777698 + }, + "runs": [ + { + "backend": "reference", + "repeat": 0, + "server_compute_s": 3.3908394610043615 + }, + { + "backend": "optimized", + "repeat": 0, + "server_compute_s": 1.877934766933322 + }, + { + "backend": "optimized", + "repeat": 1, + "server_compute_s": 1.8629616550169885 + }, + { + "backend": "reference", + "repeat": 1, + "server_compute_s": 3.6480522511992604 + }, + { + "backend": "reference", + "repeat": 2, + "server_compute_s": 3.6024102109950036 + }, + { + "backend": "optimized", + "repeat": 2, + "server_compute_s": 1.865732975071296 + }, + { + "backend": "optimized", + "repeat": 3, + "server_compute_s": 1.871090185130015 + }, + { + "backend": "reference", + "repeat": 3, + "server_compute_s": 3.708768686046824 + } + ], + "summary": { + "reference": { + "first_search_s": 3.3908394610043615, + "warm_searches_s": [ + 3.6480522511992604, + 3.6024102109950036, + 3.708768686046824 + ], + "warm_median_s": 3.6480522511992604 + }, + "optimized": { + "first_search_s": 1.877934766933322, + "warm_searches_s": [ + 1.8629616550169885, + 1.865732975071296, + 1.871090185130015 + ], + "warm_median_s": 1.865732975071296 + } + }, + "first_search_speedup": 1.8056215373985656, + "warm_median_speedup": 1.9552917271346697, + "sizes": { + "encrypted_index_bytes": 737480, + "public_keys_bytes": 85603818, + "query_bytes": 368754, + "response_bytes": 102494, + "query_plus_response_bytes": 471248 + }, + "optimized_cached_switch_keys": 11, + "optimized_packed_cache_payload_bytes": 35807568, + "cache_size_note": "sys.getsizeof of packed GMP values, including folding constants; additional to the existing public key. Excludes small Python containers and transient arithmetic allocations; not peak RSS.", + "minimum_response_noise_budget_bits": 26, + "identical_ciphertexts": true, + "all_distances_correct": true, + "top3": [ + { + "index": 0, + "distance": 0 + }, + { + "index": 31, + "distance": 0 + }, + { + "index": 19, + "distance": 241 + } + ], + "source_sha256": { + "benchmarks/bfv_server.py": "c0523cfbd4100657fc7f9eae9ac117d5fa86fc57e9983a7e2dd840920a2c2298", + "benchmarks/bfv_client_matrix.py": "d7c4c4242ad2cc537b8c39cf990944b83c3a2f178ce317e23dd415e5c12f7283", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "1a9927c6e15937ee39e6e84131890e249e14352795c21e8b8930cfe1007dd674", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "6f03ac75f42218170da2bd785605ee394940a202afdd5471545a8ae3fdc8c3d8", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "6e2c84e7882bb86f8e623c2eb2d104e0d4cab85090aed17840adc0809a5cd9a3", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "7a10c977ee648eb1975286b313f009b3508183efe9e942cd710ae6ad5ae279a3" + } +} diff --git a/benchmarks/results/native_bfv_server_profile_32.txt b/benchmarks/results/native_bfv_server_profile_32.txt new file mode 100644 index 0000000..d5ba38b --- /dev/null +++ b/benchmarks/results/native_bfv_server_profile_32.txt @@ -0,0 +1,75 @@ +Native BFV server profiles, 32 vectors x 512 bits, default parameters. +Additional warm searches from bfv_server.py, after four unprofiled searches per backend. +Timing comparisons are in native_bfv_server_32.json; cProfile timings are distorted by instrumentation. + +reference +Thu Sep 10 18:15:45 2026 /tmp/xtrace-bfv-server-profiles/reference.prof + + 19838069 function calls in 5.637 seconds + + Ordered by: internal time + List reduced from 72 to 25 due to restriction <25> + + ncalls tottime percall cumtime percall filename:lineno(function) + 262 1.335 0.005 2.976 0.011 bfv.py:39(_poly_product) + 1228950 0.468 0.000 1.338 0.000 bfv.py:300() + 21 0.431 0.021 3.992 0.190 bfv.py:411(_switch) + 2097408 0.338 0.000 0.338 0.000 bfv.py:60() + 2146566 0.303 0.000 0.303 0.000 bfv.py:56() + 268 0.271 0.001 0.271 0.001 {built-in method gmpy2.gmpy2.unpack} + 4292623 0.269 0.000 0.269 0.000 {method 'bit_length' of 'gmpy2.mpz' objects} + 524 0.249 0.000 0.968 0.002 {built-in method builtins.max} + 1245208 0.239 0.000 0.561 0.000 {built-in method _abc._abc_instancecheck} + 2146566 0.220 0.000 0.354 0.000 bfv.py:49() + 256 0.206 0.001 3.442 0.013 bfv.py:59(_ring_product) + 1245208 0.202 0.000 0.202 0.000 {built-in method _abc._abc_subclasscheck} + 1245233 0.195 0.000 0.881 0.000 {built-in method builtins.isinstance} + 1032318 0.176 0.000 0.176 0.000 bfv.py:420() + 1245208 0.125 0.000 0.686 0.000 :117(__instancecheck__) + 1245208 0.120 0.000 0.322 0.000 :121(__subclasscheck__) + 522 0.102 0.000 0.102 0.000 {built-in method gmpy2.gmpy2.pack} + 158 0.074 0.000 1.427 0.009 {built-in method builtins.any} + 311334 0.072 0.000 0.072 0.000 bfv.py:366() + 38 0.064 0.002 0.064 0.002 bfv.py:63(_automorphism) + 155667 0.040 0.000 0.040 0.000 bfv.py:501() + 19 0.019 0.001 4.117 0.217 bfv_evaluator.py:196(rotate_rows) + 2 0.019 0.010 5.502 2.751 bfv_client.py:237(_distance_tile) + 57 0.016 0.000 0.089 0.002 bfv.py:365() + 65536 0.016 0.000 0.016 0.000 bfv.py:34(_round_div) + + + +optimized +Thu Sep 10 18:15:47 2026 /tmp/xtrace-bfv-server-profiles/optimized.prof + + 3581974 function calls in 2.220 seconds + + Ordered by: internal time + List reduced from 73 to 25 due to restriction <25> + + ncalls tottime percall cumtime percall filename:lineno(function) + 21 1.293 0.062 1.426 0.068 bfv_evaluator.py:53(apply) + 1048704 0.102 0.000 0.102 0.000 bfv_evaluator.py:110() + 344106 0.071 0.000 0.071 0.000 bfv_evaluator.py:67() + 180246 0.065 0.000 0.193 0.000 bfv.py:300() + 158 0.061 0.000 0.371 0.002 {built-in method builtins.any} + 10 0.056 0.006 0.122 0.012 bfv.py:39(_poly_product) + 38 0.055 0.001 0.055 0.001 bfv.py:63(_automorphism) + 311334 0.050 0.000 0.050 0.000 bfv_evaluator.py:139() + 155667 0.039 0.000 0.039 0.000 bfv_evaluator.py:213() + 196632 0.038 0.000 0.090 0.000 {built-in method _abc._abc_instancecheck} + 58 0.035 0.001 0.035 0.001 {built-in method gmpy2.gmpy2.unpack} + 196632 0.032 0.000 0.032 0.000 {built-in method _abc._abc_subclasscheck} + 196657 0.031 0.000 0.140 0.000 {built-in method builtins.isinstance} + 148 0.028 0.000 0.057 0.000 {built-in method builtins.max} + 2 0.027 0.013 2.045 1.022 bfv_client.py:237(_distance_tile) + 144 0.025 0.000 0.025 0.000 {built-in method gmpy2.gmpy2.pack} + 196632 0.020 0.000 0.109 0.000 :117(__instancecheck__) + 42 0.019 0.000 0.111 0.003 bfv_evaluator.py:61(finish) + 130 0.019 0.000 0.019 0.000 {built-in method builtins.min} + 196632 0.019 0.000 0.051 0.000 :121(__subclasscheck__) + 57 0.016 0.000 0.066 0.001 bfv_evaluator.py:138() + 65536 0.016 0.000 0.016 0.000 bfv.py:34(_round_div) + 81930 0.013 0.000 0.013 0.000 bfv.py:56() + 163855 0.011 0.000 0.011 0.000 {method 'bit_length' of 'gmpy2.mpz' objects} + 81930 0.009 0.000 0.014 0.000 bfv.py:49() diff --git a/benchmarks/results/native_bfv_verified_1024.json b/benchmarks/results/native_bfv_verified_1024.json new file mode 100644 index 0000000..3075bde --- /dev/null +++ b/benchmarks/results/native_bfv_verified_1024.json @@ -0,0 +1,269 @@ +{ + "environment": { + "utc": "2026-09-11T04:24:09.304787+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "pycryptodome": "3.23.0", + "msgpack": "1.1.2", + "revision": "b7841e45333772d1e470939a1c2351fea17a639c", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_verified.py", + "--num-vectors", + "1024", + "--repeats", + "6", + "--json-out", + "benchmarks/results/native_bfv_verified_1024.json" + ], + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 1024, + "measurement_notes": "Identical keys and ciphertexts, independent public-only residue server contexts. Both server timers include MessagePack ciphertext decode/encode and native arithmetic; verified additionally includes authentication, session checks and outer framing. Index import, arithmetic context creation, encryption and private decryption are excluded from server times. First search has cold key/mask caches; five warm samples by default, alternating order. Client timers include ciphertext decode/decrypt/slot decode; verified adds authentication, ticket handling and private result checking. Query encryption is separately timed outside these phases. No network, remote endpoint, SEAL, security-level claim or constant-time claim.", + "setup_timings": { + "key_generation_s": 4.339769265148789, + "encrypt_index_summary_export_auth_s": 8.079525091918185, + "authenticate_import_setup_s": 1.7375480628106743, + "raw_arithmetic_context_s": 0.0016010950785130262, + "verified_arithmetic_context_s": 0.0009621409699320793, + "summary_only_preprocess_s": 0.025153255090117455 + }, + "runs": [ + { + "repeat": 0, + "phase": "raw_server", + "seconds": 2.6571030700579286 + }, + { + "repeat": 0, + "phase": "verified_server", + "seconds": 2.6737288739532232 + }, + { + "repeat": 0, + "phase": "verified_client", + "seconds": 0.041729690972715616 + }, + { + "repeat": 0, + "phase": "raw_client", + "seconds": 0.03954386292025447 + }, + { + "repeat": 1, + "phase": "verified_server", + "seconds": 2.3735971769783646 + }, + { + "repeat": 1, + "phase": "raw_server", + "seconds": 2.371556321159005 + }, + { + "repeat": 1, + "phase": "raw_client", + "seconds": 0.03905453206971288 + }, + { + "repeat": 1, + "phase": "verified_client", + "seconds": 0.04158204700797796 + }, + { + "repeat": 2, + "phase": "raw_server", + "seconds": 2.3573059749323875 + }, + { + "repeat": 2, + "phase": "verified_server", + "seconds": 2.3584984519984573 + }, + { + "repeat": 2, + "phase": "verified_client", + "seconds": 0.04113910696469247 + }, + { + "repeat": 2, + "phase": "raw_client", + "seconds": 0.03929260699078441 + }, + { + "repeat": 3, + "phase": "verified_server", + "seconds": 2.3811275989282876 + }, + { + "repeat": 3, + "phase": "raw_server", + "seconds": 2.3726596529595554 + }, + { + "repeat": 3, + "phase": "raw_client", + "seconds": 0.03970337612554431 + }, + { + "repeat": 3, + "phase": "verified_client", + "seconds": 0.04124582908116281 + }, + { + "repeat": 4, + "phase": "raw_server", + "seconds": 2.3679882378783077 + }, + { + "repeat": 4, + "phase": "verified_server", + "seconds": 2.3805392258800566 + }, + { + "repeat": 4, + "phase": "verified_client", + "seconds": 0.041710648918524384 + }, + { + "repeat": 4, + "phase": "raw_client", + "seconds": 0.03959750197827816 + }, + { + "repeat": 5, + "phase": "verified_server", + "seconds": 2.373485399177298 + }, + { + "repeat": 5, + "phase": "raw_server", + "seconds": 2.367954927030951 + }, + { + "repeat": 5, + "phase": "raw_client", + "seconds": 0.03955245087854564 + }, + { + "repeat": 5, + "phase": "verified_client", + "seconds": 0.04190468299202621 + } + ], + "summary": { + "raw_server": { + "first_s": 2.6571030700579286, + "warm_samples_s": [ + 2.371556321159005, + 2.3573059749323875, + 2.3726596529595554, + 2.3679882378783077, + 2.367954927030951 + ], + "warm_median_s": 2.3679882378783077 + }, + "verified_server": { + "first_s": 2.6737288739532232, + "warm_samples_s": [ + 2.3735971769783646, + 2.3584984519984573, + 2.3811275989282876, + 2.3805392258800566, + 2.373485399177298 + ], + "warm_median_s": 2.3735971769783646 + }, + "raw_client": { + "first_s": 0.03954386292025447, + "warm_samples_s": [ + 0.03905453206971288, + 0.03929260699078441, + 0.03970337612554431, + 0.03959750197827816, + 0.03955245087854564 + ], + "warm_median_s": 0.03955245087854564 + }, + "verified_client": { + "first_s": 0.041729690972715616, + "warm_samples_s": [ + 0.04158204700797796, + 0.04113910696469247, + 0.04124582908116281, + 0.041710648918524384, + 0.04190468299202621 + ], + "warm_median_s": 0.04158204700797796 + } + }, + "query_encrypt_and_bind_samples_s": [ + 0.10001991898752749, + 0.09990808507427573, + 0.09966578800231218, + 0.09928345982916653, + 0.10008095717057586, + 0.09987170086242259 + ], + "summary_only_check_samples_s": [ + 0.002144736936315894, + 0.0021136258728802204, + 0.0021337070502340794, + 0.0021290569566190243, + 0.002179748145863414, + 0.002148417057469487 + ], + "protect_private_state_s": 0.0023410909343510866, + "sizes": { + "public_keys_bytes": 85603219, + "index_ciphertext_array_bytes": 23598467, + "authenticated_setup_bytes": 109205980, + "raw_query_array_bytes": 368727, + "raw_response_array_bytes": 102467, + "authenticated_query_bytes": 368840, + "authenticated_response_bytes": 102614, + "authenticated_query_plus_response_bytes": 471454, + "query_authentication_and_framing_overhead_bytes": 113, + "response_authentication_and_framing_overhead_bytes": 147, + "private_verification_payload_bytes": 16448, + "protected_private_state_bytes": 36729, + "previous_benchmark_encrypted_index_bytes": 23598496, + "previous_benchmark_query_bytes": 368754, + "previous_benchmark_response_bytes": 102496 + }, + "identical_ciphertexts": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_verified.py": "1578cf1c019159a9856bc31d0f3561499ce10b5d024eb2d24682f14a79b6884e", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "e781be6172b9f1ad676fa4ac5f706be7035563bc230e285a56b90129bf18b654", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "05e30fc3b8191e2fd0f0620ac7dbbc35ccea0a43913fc62f66575b0d44716aeb", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } +} diff --git a/benchmarks/results/native_bfv_verified_32.json b/benchmarks/results/native_bfv_verified_32.json new file mode 100644 index 0000000..84e93e7 --- /dev/null +++ b/benchmarks/results/native_bfv_verified_32.json @@ -0,0 +1,269 @@ +{ + "environment": { + "utc": "2026-09-11T04:24:37.184175+00:00", + "python": "3.12.3", + "platform": "Linux-7.0.0-30-generic-x86_64-with-glibc2.39", + "gmpy2": "2.3.0", + "gmp": "GMP 6.3.0", + "pycryptodome": "3.23.0", + "msgpack": "1.1.2", + "revision": "b7841e45333772d1e470939a1c2351fea17a639c", + "working_tree_dirty": true + }, + "command": [ + "benchmarks/bfv_verified.py", + "--num-vectors", + "32", + "--repeats", + "6", + "--json-out", + "benchmarks/results/native_bfv_verified_32.json" + ], + "config": { + "embed_len": 512, + "poly_modulus_degree": 8192, + "plain_modulus": 65537, + "coeff_modulus_bits": 180, + "decomposition_bits": 30, + "error_eta": 21, + "rns_modulus": true, + "response_modulus_bits": 50, + "device": "cpu" + }, + "seed": 1337, + "vectors": 32, + "measurement_notes": "Identical keys and ciphertexts, independent public-only residue server contexts. Both server timers include MessagePack ciphertext decode/encode and native arithmetic; verified additionally includes authentication, session checks and outer framing. Index import, arithmetic context creation, encryption and private decryption are excluded from server times. First search has cold key/mask caches; five warm samples by default, alternating order. Client timers include ciphertext decode/decrypt/slot decode; verified adds authentication, ticket handling and private result checking. Query encryption is separately timed outside these phases. No network, remote endpoint, SEAL, security-level claim or constant-time claim.", + "setup_timings": { + "key_generation_s": 4.338192637078464, + "encrypt_index_summary_export_auth_s": 1.3449218550231308, + "authenticate_import_setup_s": 1.6677678199484944, + "raw_arithmetic_context_s": 0.0015940847806632519, + "verified_arithmetic_context_s": 0.0009516510181128979, + "summary_only_preprocess_s": 0.0008359679486602545 + }, + "runs": [ + { + "repeat": 0, + "phase": "raw_server", + "seconds": 0.3808726631104946 + }, + { + "repeat": 0, + "phase": "verified_server", + "seconds": 0.38237067707814276 + }, + { + "repeat": 0, + "phase": "verified_client", + "seconds": 0.03770374902524054 + }, + { + "repeat": 0, + "phase": "raw_client", + "seconds": 0.03787448303773999 + }, + { + "repeat": 1, + "phase": "verified_server", + "seconds": 0.07546799909323454 + }, + { + "repeat": 1, + "phase": "raw_server", + "seconds": 0.07493602787144482 + }, + { + "repeat": 1, + "phase": "raw_client", + "seconds": 0.037949763936921954 + }, + { + "repeat": 1, + "phase": "verified_client", + "seconds": 0.03785317204892635 + }, + { + "repeat": 2, + "phase": "raw_server", + "seconds": 0.07507768110372126 + }, + { + "repeat": 2, + "phase": "verified_server", + "seconds": 0.07534101605415344 + }, + { + "repeat": 2, + "phase": "verified_client", + "seconds": 0.03754223510622978 + }, + { + "repeat": 2, + "phase": "raw_client", + "seconds": 0.037920583970844746 + }, + { + "repeat": 3, + "phase": "verified_server", + "seconds": 0.07540737697854638 + }, + { + "repeat": 3, + "phase": "raw_server", + "seconds": 0.07481546490453184 + }, + { + "repeat": 3, + "phase": "raw_client", + "seconds": 0.037693017860874534 + }, + { + "repeat": 3, + "phase": "verified_client", + "seconds": 0.03777332999743521 + }, + { + "repeat": 4, + "phase": "raw_server", + "seconds": 0.07513878191821277 + }, + { + "repeat": 4, + "phase": "verified_server", + "seconds": 0.07539898692630231 + }, + { + "repeat": 4, + "phase": "verified_client", + "seconds": 0.03783954097889364 + }, + { + "repeat": 4, + "phase": "raw_client", + "seconds": 0.03770379885099828 + }, + { + "repeat": 5, + "phase": "verified_server", + "seconds": 0.07531027519144118 + }, + { + "repeat": 5, + "phase": "raw_server", + "seconds": 0.07499737804755569 + }, + { + "repeat": 5, + "phase": "raw_client", + "seconds": 0.03793373401276767 + }, + { + "repeat": 5, + "phase": "verified_client", + "seconds": 0.03781877108849585 + } + ], + "summary": { + "raw_server": { + "first_s": 0.3808726631104946, + "warm_samples_s": [ + 0.07493602787144482, + 0.07507768110372126, + 0.07481546490453184, + 0.07513878191821277, + 0.07499737804755569 + ], + "warm_median_s": 0.07499737804755569 + }, + "verified_server": { + "first_s": 0.38237067707814276, + "warm_samples_s": [ + 0.07546799909323454, + 0.07534101605415344, + 0.07540737697854638, + 0.07539898692630231, + 0.07531027519144118 + ], + "warm_median_s": 0.07539898692630231 + }, + "raw_client": { + "first_s": 0.03787448303773999, + "warm_samples_s": [ + 0.037949763936921954, + 0.037920583970844746, + 0.037693017860874534, + 0.03770379885099828, + 0.03793373401276767 + ], + "warm_median_s": 0.037920583970844746 + }, + "verified_client": { + "first_s": 0.03770374902524054, + "warm_samples_s": [ + 0.03785317204892635, + 0.03754223510622978, + 0.03777332999743521, + 0.03783954097889364, + 0.03781877108849585 + ], + "warm_median_s": 0.03781877108849585 + } + }, + "query_encrypt_and_bind_samples_s": [ + 0.10028813499957323, + 0.0998352849856019, + 0.0996188500430435, + 0.10010351985692978, + 0.10015947092324495, + 0.09966332116164267 + ], + "summary_only_check_samples_s": [ + 0.0003099560271948576, + 0.00028055603615939617, + 0.00035280780866742134, + 0.00031211692839860916, + 0.00031296699307858944, + 0.00032019708305597305 + ], + "protect_private_state_s": 0.0022762599401175976, + "sizes": { + "public_keys_bytes": 85603214, + "index_ciphertext_array_bytes": 737453, + "authenticated_setup_bytes": 86341035, + "raw_query_array_bytes": 368727, + "raw_response_array_bytes": 102467, + "authenticated_query_bytes": 368840, + "authenticated_response_bytes": 102614, + "authenticated_query_plus_response_bytes": 471454, + "query_authentication_and_framing_overhead_bytes": 113, + "response_authentication_and_framing_overhead_bytes": 147, + "private_verification_payload_bytes": 16448, + "protected_private_state_bytes": 36713, + "previous_benchmark_encrypted_index_bytes": 737480, + "previous_benchmark_query_bytes": 368754, + "previous_benchmark_response_bytes": 102494 + }, + "identical_ciphertexts": true, + "all_distances_correct": true, + "source_sha256": { + "benchmarks/bfv_verified.py": "1578cf1c019159a9856bc31d0f3561499ce10b5d024eb2d24682f14a79b6884e", + "benchmarks/bfv_client_matrix.py": "4962c7cf837a67becb240e48c7af7068def6276f6bf9647d518de3735be58a89", + "src/xtrace_sdk/x_vec/crypto/bfv_client.py": "becd81b9c41c027fe26de3e675adfba32fae7c0e6b0e1bede73f96a14effc775", + "src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py": "e781be6172b9f1ad676fa4ac5f706be7035563bc230e285a56b90129bf18b654", + "src/xtrace_sdk/x_vec/crypto/bfv_security.py": "17149c07384889f994f93bc9e7b209d965176cd916c97a0f6b7a138861a7204e", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv.py": "51eee672c2d0514146f97c9ddf33c7f4cfb99114a2b7d86654ec7de7422ebca7", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_native.py": "13b0028e889351ade4d9dff2f5605e969c015cebe890aa80fd5489593d4ef0cf", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py": "74663e4b22d2fc0128ad2572cb10f9d3727b404fc0240661b08d308ba57cad40", + "src/xtrace_sdk/x_vec/crypto/encryption/bfv_evaluator.py": "895fc201b641a1ea5242b0c54a001d9402afcfb4d21668ca33a8b9992b4132dc", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp": "da003cf5e6553f85fa402c943a762504d661fb16657d25b16116f3f667509cae", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/profile.h": "3a98bb8c322dec75d1c3ed2531767ac1f5b43a1d7e79a9927d47052d43541252", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h": "e77f270c620d432370e8204c24e1dc6f3c0d382d6bd777ad63852dd8a89f575a", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h": "70f96bc7f1fc25827979cea7b350edaf9cffccf2fa5af0a9275b9d514fb400cf", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_scale.h": "3f191dc1532afe2e8774279d251d3374b34dc8b40b4e8a97d3dc341959ab89d1", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/packed_wire.h": "0fe06f2c6cb333ef90cf0b66847815dbf521e81be881ffd55aff21009cde42fd", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h": "5f46b7490d556a2f314dbb5cda6f022cacac6f952f3eda48640e068f352e3796", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/_bfv_rns.cpython-312-x86_64-linux-gnu.so": "539e19f0591c9dd375a7cf519569c4777067fb6593f5157d87b7edb37a42eb04", + "src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/Makefile": "05e30fc3b8191e2fd0f0620ac7dbbc35ccea0a43913fc62f66575b0d44716aeb", + "src/xtrace_sdk/x_vec/utils/xtrace_types.py": "3d4daa19d5d9d536573b2e2c65aa13d9559a4bc5e6ae01e2b5ffdeeac8bae962" + } +} diff --git a/docs/api_rst/xtrace_sdk.crypto.encryption.rst b/docs/api_rst/xtrace_sdk.crypto.encryption.rst index 9876089..465b3b8 100644 --- a/docs/api_rst/xtrace_sdk.crypto.encryption.rst +++ b/docs/api_rst/xtrace_sdk.crypto.encryption.rst @@ -4,6 +4,38 @@ xtrace\_sdk.crypto.encryption package Submodules ---------- +xtrace\_sdk.crypto.encryption.bfv module +--------------------------------------- + +.. automodule:: xtrace_sdk.x_vec.crypto.encryption.bfv + :members: + :show-inheritance: + :undoc-members: + +xtrace\_sdk.crypto.encryption.bfv\_evaluator module +------------------------------------------------- + +.. automodule:: xtrace_sdk.x_vec.crypto.encryption.bfv_evaluator + :members: + :show-inheritance: + :undoc-members: + +xtrace\_sdk.crypto.encryption.bfv\_rns module +------------------------------------------- + +.. automodule:: xtrace_sdk.x_vec.crypto.encryption.bfv_rns + :members: + :show-inheritance: + :undoc-members: + +xtrace\_sdk.crypto.encryption.bfv\_native module +---------------------------------------------- + +.. automodule:: xtrace_sdk.x_vec.crypto.encryption.bfv_native + :members: + :show-inheritance: + :undoc-members: + xtrace\_sdk.crypto.encryption.aes module ---------------------------------------- diff --git a/docs/api_rst/xtrace_sdk.crypto.rst b/docs/api_rst/xtrace_sdk.crypto.rst index c00b411..645ea45 100644 --- a/docs/api_rst/xtrace_sdk.crypto.rst +++ b/docs/api_rst/xtrace_sdk.crypto.rst @@ -14,6 +14,14 @@ Subpackages Submodules ---------- +xtrace\_sdk.crypto.bfv\_client module +------------------------------------ + +.. automodule:: xtrace_sdk.x_vec.crypto.bfv_client + :members: + :show-inheritance: + :undoc-members: + xtrace\_sdk.crypto.goldwasser\_micali\_client module ---------------------------------------------------- diff --git a/docs/index.rst b/docs/index.rst index c407d3c..b4eac81 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -77,6 +77,23 @@ Choose your path: manual_rst/changelog manual_rst/contribute +.. toctree:: + :maxdepth: 1 + :caption: Research + + research/native-bfv + research/native-bfv-ntt + research/native-bfv-server + research/native-bfv-residue + research/native-bfv-fused-rns + research/native-bfv-security + research/native-bfv-verified-sessions + research/native-bfv-predecryption + research/native-bfv-verification-plan + research/native-bfv-nitro + research/native-bfv-code-audit + research/bfv-packed-hamming + .. toctree:: :maxdepth: 1 :caption: API Reference diff --git a/docs/research/bfv-packed-hamming.md b/docs/research/bfv-packed-hamming.md new file mode 100644 index 0000000..7b4c4bb --- /dev/null +++ b/docs/research/bfv-packed-hamming.md @@ -0,0 +1,358 @@ +# BFV for XTrace: repository analysis and first experiment + +Research date: 2026-09-08. Experimental branch: `research/bfv-packed-hamming`, +based on `staging-GPU-paillier-client` at `c8fc6838aec81e2c3a05fbd5c0c9c0e8635beb26`. +The local `main` reference used for comparison is +`0f8522e449d00622e10f4111ac8eb89d52ca577b`. No remote refresh was performed. + +**BFV can substantially reduce the distance-response payload.** The implemented +experiment computes distances under BFV and packs them before returning them. +For 8,192 binary vectors of dimension 512, the measured response fell from +4.26 MB to 127 KB including IDs and framing, a 33.4× reduction. Query plus response +fell by 7.6×. The cost was an approximately 52× larger encrypted index and a +39.6 MB public/evaluation-key bundle. These are local measurements, not a production +service or GPU performance claim. + +Packing still returns all candidate distances. Returning only the top three +requires a separate encrypted comparison and selection protocol. No such protocol +is implemented here. Existing Paillier ciphertexts are not automatically usable +as BFV ciphertexts. + +## What the repository implements + +The repository is the Python **client SDK**, with public arithmetic helpers and +CUDA client extensions. It does not contain the deployed search server. The +experiment therefore supplies a local server evaluator rather than changing the +live API. + +| Area | Responsibility and relevance | +| --- | --- | +| `src/xtrace_sdk/x_vec/inference/embedding.py` | Produces embeddings through Sentence Transformers, Ollama, or OpenAI. `float_2_bin` thresholds each coordinate with `> 0`. Search operates on those binary vectors, not the original floating-point vectors. Provider selection determines where embedding plaintext is processed. | +| `x_vec/data_loaders/loader.py` | AES-encrypts document text, binarizes embeddings, encrypts the binary index through the homomorphic client, and uploads via the integration. Supports per-vector and batch encryption. Metadata is supplied separately. | +| `x_vec/crypto/encryption/paillier.py` | Standard additive Paillier over plaintexts modulo `n`, with ciphertext multiplication modulo `n²` implementing plaintext addition. `key_len` is the bit length of **each prime**. | +| `x_vec/crypto/encryption/paillier_lookup.py` | Custom variant with an alternative decryption exponent `a`, tables for powers of `g` in 8-bit message chunks, and a randomization table. Its parameter/security analysis is separate from this BFV experiment. | +| `x_vec/crypto/{paillier_client,paillier_lookup_client}.py` | Packs bits with zero separators, dispatches CPU/GPU encryption and decryption, serializes keys/configuration, and supplies an offline public-key Hamming-encoding helper. | +| `x_vec/crypto/*_gpu_ext/` | CUDA/GMP/pybind11 implementations using CGBN for big integers, batched modular operations, and decoding/popcount. The lookup implementation also caches tables and exposes decrypt/re-encrypt helpers; those helpers require secret material and cannot simply move to an untrusted server. | +| `x_vec/crypto/device.py` | Chooses the requested CPU/GPU implementation or probes for automatic selection. It changes execution, not the result format or candidate count. | +| `x_vec/crypto/encryption/aes.py`, `crypto/key_provider.py` | AES-256-GCM for contents and protected secret-key storage; scrypt passphrases or AWS KMS envelope keys. BFV does not need to replace content encryption. | +| `x_vec/utils/execution_context.py` | Bundles the homomorphic client and AES/key-provider state, serializes/restores it locally or remotely, and enforces the embedding/key-size constraints. Current supported contexts remain Paillier and Paillier-Lookup. | +| `integrations/xtrace.py` | Async HTTP API for chunks, metadata, contexts, knowledge bases and administration. Queries use base64 ciphertexts in JSON. Distance responses are read completely and decoded as MessagePack or JSON. | +| `x_vec/retrievers/retriever.py` | Encrypts queries, downloads candidate encodings, decrypts/decodes every distance, then calls `np.argsort(... )[:k]`. A second request fetches selected chunks for AES decryption. Optional multiprocessing changes client decoding, not the wire volume. | +| `cli/` | Initialization/environment and cached session state; file chunking; load/upsert/fetch/retrieve and knowledge-base commands; interactive shell. These routes ultimately use the same SDK clients and integration. | +| `x_vec/inference/llm.py` | Optional RAG inference using retrieved plaintext context. Provider choice defines its separate plaintext boundary. | +| Other crypto / `x_mem` | Goldwasser–Micali is a separate XOR-capable implementation; Merkle commitments are a partial utility and do not provide search-result verification. Signatures and `x_mem` are placeholders. | +| `tests/` | Offline encryption, embedding conversion, execution-context/key persistence and CLI tests; separate live API tests; CPU/GPU interoperability tests. | +| `docs/`, `examples/` | Sphinx API/manual pages and a quick-start notebook documenting the same client workflow. | +| `pyproject.toml`, `hatch_build.py`, `build_gpu_binaries.sh`, `.github/workflows/` | Hatch/VCS packaging, CUDA builds and binary wheel tagging, lint/type checks, offline tests, gated GPU/integration tests and release/docs workflows. | + +Paths abbreviated to `x_vec/...` in the table are relative to `src/xtrace_sdk/`. +The existing untracked `benchmarks/paillier_client_matrix.py` is preserved. + +The branch comparison shows the GPU work primarily changes client dispatch, +CUDA implementations, configuration, packaging and tests. The primitive encryption +modules, retriever and integration are identical between the recorded `main` and +starting GPU-branch commits. Their response protocol has not been changed by the +GPU work. + +## The current arithmetic and I/O bottleneck + +For a binary vector `x`, the client effectively encodes a base-four integer: + +```text +P(x) = sum_j x[j] * 4^(d - 1 - j) +``` + +Each base-four digit is represented by the binary pair `0x[j]`. Adding two such +encodings produces digits in `{0, 1, 2}` without carries between coordinates. +The low bit of each digit is `x[j] XOR q[j]`. Counting those low bits after +decryption gives the Hamming distance. + +The server can obtain `Enc(P(x) + P(q))` through Paillier ciphertext multiplication. +It cannot use Paillier's additive operation alone to evaluate the nonlinear bit +extraction and popcount that the SDK currently performs after decryption. +Consequently, the returned encrypted object encodes coordinate-wise sums, not a +single scalar distance already computed on the server. + +`XTraceIntegration.compute_hamming_distances` awaits `res.read()` and parses the +whole response. `Retriever.nn_search_for_ids` then decodes all candidate results +and ranks them. There is no `k` argument in the distance API request. Reducing +`k` from ten to three does not reduce this response. + +The payload consists of one encrypted **result per candidate** (possibly multiple +chunks for larger vectors), not all AES-encrypted documents. Metadata/range +filters can restrict the candidate set first. Only the selected documents are +fetched afterwards. Thus the relevant scaling variable is the number of candidates +after filtering, which can be much smaller than the whole database. + +At the benchmark defaults, `key_len=1024` means approximately 2,048-bit `n`, so +each ciphertext modulo `n²` occupies about 512 binary bytes. GPU execution and +lookup tables retain that ciphertext representation. Streaming the existing +response could overlap download and decryption and reduce memory, but would not +remove these bytes. + +## What BFV adds, and what “put Paillier inside BFV” could mean + +BFV supports exact modular-integer addition and multiplication. With batch +encoding, one ciphertext holds many independent integer slots. SEAL's BFV +batching has `N` slots arranged as two rows of `N/2`; a prime plaintext modulus +congruent to `1 mod 2N` enables that encoding. +[SEAL batch-encoding example](https://github.com/microsoft/SEAL/blob/main/native/examples/2_encoders.cpp) + +For binary vectors, the distance is the low-depth arithmetic expression + +```text +h_i = sum_j (x[i,j] - q[j])² + = sum_j (x[i,j] + q[j] - 2*x[i,j]*q[j]). +``` + +The first form needs one encrypted subtraction and square per packed input, +followed by encrypted sums. Both database and query remain encrypted. If the +plaintext modulus exceeds `d` and the noise budget remains positive, decoding +gives the exact integer distance. This experiment uses `t=65537` and `d<=4096`. + +There are three distinct interpretations of the mentor's suggestion: + +| Interpretation | Assessment | +| --- | --- | +| Encrypt the bytes/limbs of existing Paillier ciphertexts inside BFV | This retains the Paillier representation and adds an encryption layer. It does not itself extract the small distance or make randomized ciphertext bytes compressible. | +| Evaluate Paillier decryption and distance decoding homomorphically under BFV | Possible in principle with appropriately encrypted secret-key material and a sufficiently capable circuit. It requires large-integer modular arithmetic and bit extraction; it is not ordinary BFV key switching. No practical implementation of this conversion was established in this investigation. | +| Compute the distances directly under BFV, then pack results from many input ciphertexts | Implemented here. This is a plausible interpretation of the demo, but cannot be identified as the actual demo without its code. It requires BFV-encrypted index data. | + +These distinctions follow from the two schemes' plaintext representations and +available operations; they are design conclusions, not a claim that all possible +hybrid protocols have been ruled out. An interactive conversion protocol is also +a separate avenue and would need its own trust, leakage and round-trip analysis. +The textbook's BFV key-switching discussion concerns changing the secret key of +an RLWE ciphertext, not a turnkey Paillier-to-BFV conversion. + +## The implemented packing layout + +Let `M` be the number of candidates, `d` the embedding dimension, `N=8192` the +slot count, and `D` the next power of two at least `d`. Define +`B=N/(2D)` lanes in each BFV row and `C=2B` records per input ciphertext. +At `d=512`, `D=512`, `B=8`, and `C=16`. + +Each input tile arranges a BFV row as follows; the second row holds another +eight candidates: + +```text +row 0: [dimension 0 of candidates 0..7] + [dimension 1 of candidates 0..7] + ... + [dimension 511 of candidates 0..7] + +row 1: the same arrangement for candidates 8..15 +``` + +The query repeats each bit across the eight lanes, then repeats that row. +The same one-ciphertext query is reused for every tile. This avoids sending +one full BFV ciphertext per query dimension. + +1. Subtract query from encrypted tile and square slotwise. Relinearize to two + ciphertext polynomials. +2. Rotate rows by `B, 2B, 4B, ...` and add. This sums across dimensions while + keeping different candidate lanes separate. It takes `log2(D)` rotations. +3. Apply a public plaintext mask, keeping only the first `B` distances in each + row and zeroing invalid candidates in the final tile. +4. Merge results from different ciphertexts with rotations and additions. A + binary carry tree uses one rotation per merge and logarithmic intermediate + ciphertext memory. It handles incomplete groups as well as full groups. +5. After up to `D` tiles, the output holds up to `N` distances. Drop unused + coefficient-modulus primes before serialization. + +This reduction uses approximately `ceil(M/C)` input ciphertexts and +`ceil(M/N)` response ciphertexts. For a full 8,192-record, 512-dimensional group, +there are 512 input ciphertexts, 512 squares, 4,608 dimension-sum rotations, +511 packing rotations, and one returned ciphertext. The whole index packet is +still materialized in memory; the logarithmic bound applies to intermediate +arithmetic, not index storage or MessagePack parsing. + +Rows rotate independently in BFV. The code keeps both rows separate and decodes +their known layout back to original record order. It does not assume a flat +8,192-slot circular rotation. This matches SEAL's +[rotation model](https://github.com/microsoft/SEAL/blob/main/native/examples/6_rotation.cpp). + +The server uses fixed, SEAL-validated `TC128` parameters. Final modulus switching +reduces ciphertext size while consuming remaining computation capacity. It happens +only after packing, when no further encrypted arithmetic is needed. +[SEAL's levels example](https://github.com/microsoft/SEAL/blob/main/native/examples/3_levels.cpp) + +`noise_probe.py` demonstrates why simply halving the ring is unsuitable for this +particular circuit and parameter set. In the 4,096-slot trial, coefficient-prime +sizes were `[36,36,37]`: remaining noise was about 21 bits after the square, 13 +after summation, and **zero after masking**. The 8,192-slot version retained ample +budget at that stage. Exact numbers vary with fresh encryption randomness. +This is a failure of that tested configuration, not a proof that every possible +4,096-slot Hamming design fails. The main prototype excludes that configuration. + +## Measurements and interpretation + +The raw [measurement records](../../experiments/bfv/results/measurements.jsonl) +contain seed, dependency versions, source hashes, exact byte counts, phase timings +and the baseline commit. These runs use 512-dimensional synthetic binary vectors, +sequential integer IDs, `k=3`, standard Paillier with 1,024-bit primes, and one +query per newly built index. Encryption randomness is fresh; only the synthetic +plaintext data is seeded. Every decoded distance and selected ID was checked. + +Sizes below use decimal KB/MB. **Response sizes include IDs and MessagePack framing.** +For a full BFV block, the ciphertext itself is about 103 KB; the remainder is +primarily candidate IDs. Real ID widths and transport framing affect the ratio. + +| Candidates | Paillier response | BFV response | Response reduction | Query + response reduction | +| ---: | ---: | ---: | ---: | ---: | +| 256 | 132.7 KB | 103.6 KB | 1.28× | 0.25×: BFV transfers about 4× more | +| 2,048 | 1.065 MB | 109.0 KB | 9.77× | 1.97× | +| 8,192 | 4.259 MB | 127.4 KB | 33.43× | 7.61× | + +The BFV query is approximately 432.6 KB in all these runs; the modeled current +Paillier JSON/base64 query is 798 bytes. Ignoring setup, the first BFV result block +therefore needs roughly a thousand candidates before **bidirectional** byte +savings begin. The much lower download-only crossover is not enough to establish +total I/O savings. + +For 8,192 candidates: + +| Cost | Paillier | BFV | +| --- | ---: | ---: | +| Binary index model | 4.26 MB | 221.44 MB | +| Public/evaluation setup | About 2.5 KB | 39.59 MB | +| Local server evaluation + response serialization | 0.105 s | 20.093 s | +| Local client response decoding + stable top-k | 56.958 s | 0.012 s | + +The BFV index is about 52× larger. Each input ciphertext holds only 16 vectors; +packing the **results** more tightly does not remove the storage needed for the +input coordinates. At this candidate count, about 3.70 MB are saved per query. +Byte-only amortization takes approximately 11 queries for the extra key bundle, +or 70 queries including the extra index. At 2,048 candidates those counts are +approximately 76 and 180. These calculations use binary Paillier index storage; +the actual JSON/base64 ingest protocol adds further overhead. + +These timing figures are illustrative local CPU measurements, not controlled +service benchmarks. They include the prototype's file-based serialization and +input parsing; they exclude actual network transfer, HTTP/TLS, authentication, +embeddings, AES content fetch, and multi-user contention. The large run overlapped +part of the existing test execution, so use isolated repeated runs for timing +decisions. Byte counts and correctness checks are unaffected. The baseline uses +the recorded `main` CPU cryptography and a local reference server operation, +with the same stable client top-k routine on both paths; it is not a replay of +the live service or its exact sorting implementation. + +**Do not infer a speedup over the optimized GPU/lookup branch from this table.** +Its decryption cost may be much smaller than this CPU baseline. BFV adds server +work; at a fast network link that extra work can outweigh the reduced traffic. +A useful next timing comparison is: + +```text +latency = query encryption + server evaluation + client decoding/selection + + query_bytes / upload_bytes_per_second + + response_bytes / download_bytes_per_second + + common round-trip overhead +``` + +Apply that model with the intended GPU server/client hardware, actual candidate +counts, index update rate, and asymmetric link bandwidth. Amortize setup and +index costs separately. The experimental code does not automatically choose a +backend from these estimates. + +## Boundaries and next steps + +The implementation and commands are in the +[experiment README](https://github.com/XTraceAI/xtrace-sdk/blob/46b637b604e1c2835653140610b7b2396f10b284/experiments/bfv/README.md). Production source and +dependencies remain unchanged. The generated assets contain measurements only; +no document data, API credentials, ciphertext indexes or secret keys are committed. + +For a usable BFV option, the next steps are: + +1. **Reconcile this layout with Liwen's demo.** Determine whether it used native + BFV distances, a hybrid conversion, or encrypted selection. Its parameters, + trust model, input format and measured bytes matter more than the scheme name. +2. **Benchmark the real workload and hardware.** Compare standard Paillier and + Paillier-Lookup with their effective parameter settings, include GPU decode, + and measure update/storage costs and amortized key transfer. The current + measurements compare configurations, not equal-security cryptosystems. +3. **Introduce a separate BFV index and endpoint.** The existing `EncryptedVector` + and `HammingClientBase` interfaces model per-record integer ciphertext lists. + Packed BFV tiles and multi-distance responses need an explicit, versioned + interface rather than changing the meaning of those existing methods. +4. **Reindex on the client.** Use retained binary vectors, regenerated local + embeddings, or client-side decryption of existing Paillier index encodings. + Keep the AES content and record IDs, and retain a Paillier index for fallback. + An existing Paillier-only knowledge base cannot answer a BFV query as-is. +5. **Implement tile-aware storage, updates and filtering.** Selecting arbitrary + IDs cannot simply relabel packed slots. Metadata/range filters need matching + lane masks and result compaction or prepartitioned indexes. A single-record + update may require rewriting a tile or a separately designed update protocol. + +If the goal is strictly **only three encrypted results on the wire**, add a +separate encrypted top-k experiment after packed distance evaluation. Exact +comparisons are not a native cheap BFV operation: a sorting/selection network +needs comparison circuits plus conditional moves, increasing depth and server +cost. IDs may need multiple limbs because real 64-bit IDs do not fit in one slot +modulo 65,537. Tie handling and whether the server learns selected IDs must be +specified. A final ordinary fetch by client-selected ID reveals an access pattern, +as it does in the current SDK. + +OpenFHE provides comparison/argmin examples through CKKS↔FHEW scheme switching. +That is a concrete direction to investigate, not a drop-in Paillier→BFV or +BFV→top-k conversion. Integer ranking correctness and conversion boundaries would +need testing. [OpenFHE scheme-switching example](https://github.com/openfheorg/openfhe-development/blob/main/src/pke/examples/scheme-switching.cpp) + +The threat model for this prototype is a single key owner and honest evaluation. +The server sees IDs, count and dimension, and receives encrypted vectors/query +plus public evaluation keys. Key-ID/schema checks catch accidental mismatches; +they do not authenticate messages, bind responses to queries, prevent replay or +prove correct computation. Noise and decrypted-result diagnostics remain local. +These protocol limits follow SEAL's documented distinctions around authenticity, +circuit privacy and decryption feedback. +[SEAL security guidance](https://github.com/microsoft/SEAL/blob/main/SECURITY.md) + +There is also a concrete dependency issue before any deployment: Microsoft now +recommends SEAL 4.4.0 or later for fixes involving untrusted inputs and native +memory safety. The convenient TenSEAL 0.3.16 experiment wheel must not be treated +as evidence of an updated native backend. Use a verified, maintained backend +before exposing the evaluator or deserializer to service traffic. +[SEAL update notice](https://github.com/microsoft/SEAL) + +## Reading guide for the provided textbook + +The supplied PDF is Ronny Ko's *The Beginner's Textbook for Fully Homomorphic +Encryption*, arXiv:2503.05136v26, dated 2026-06-02. Its relevant **printed** pages +and sections are: + +| Section | Printed page | Why it matters here | +| --- | ---: | --- | +| D-2.2, Batch Encoding | 137 | Explains SIMD integer slots and encoding. | +| D-2.3, Encryption and Decryption | 139 | Explains the plaintext/noise distinction and correctness conditions. | +| D-2.4–D-2.7, Arithmetic | 140–149 | Covers addition, plaintext masking, ciphertext multiplication and relinearization. | +| D-2.8, Key Switching | 151 | Clarifies that key switching is within the RLWE construction. | +| D-2.9, Slot Rotation | 151–163 | Supplies the operation used for dimension sums and result packing. | +| D-2.11, Bootstrapping | 164 onward | Useful for deeper circuits; unnecessary for this one-square distance circuit. | + +The book's [online BFV chapter](https://fhetextbook.github.io/BFVScheme.html) +is also available. For implementation details, use the SEAL batching, rotation +and modulus-level examples linked at the relevant points above; the low-level +Python bindings are exposed by +[TenSEAL's `sealapi`](https://github.com/OpenMined/TenSEAL/blob/main/tenseal/sealapi/__init__.py). + +## Validation recorded for this branch + +The BFV suite passed all 28 tests, including a separate-process server, full and +partial output groups, both batching rows, dimension padding, exact matches, +complements, ties, randomized encryption, malformed messages, and modulus +compaction. The three benchmark sizes also checked every distance and top-k. + +The final combined run passed **82 tests**: 28 BFV tests and 54 existing CPU/CLI +tests. It excluded the live API modules and the GPU test module: + +```bash +.venv/bin/python -m pytest experiments/bfv tests/cli tests/x_vec \ + --ignore=tests/x_vec/test_hamming_search.py \ + --ignore=tests/x_vec/test_meta_search.py \ + --ignore=tests/x_vec/test_GPU_paillier_clients.py -q +``` + +Ruff and mypy passed for the production source and the typed experiment code. +The initial existing offline regression run had 56 passes and seven GPU failures: +the compiled extensions import, but encryption raises `cudaErrorNoDevice`, and +`nvidia-smi` cannot communicate with the driver. This predates the experiment; +no production source was modified. Live API tests were not run. diff --git a/docs/research/native-bfv-code-audit.md b/docs/research/native-bfv-code-audit.md new file mode 100644 index 0000000..4f1f921 --- /dev/null +++ b/docs/research/native-bfv-code-audit.md @@ -0,0 +1,170 @@ +# BFV code audit before staging review + +Date: 2026-09-15. Branch: `research/bfv-packed-hamming`. +Comparison target: `origin/staging` at `d94ddf0`. +The previous committed BFV baseline is `46b637b`; the Nitro implementation and +the fixes described here are included in the staging pull request. + +This internal audit reviewed readability, organization, correctness and test +coverage across the BFV work. It found and fixed integration and error-handling +defects. It is a bounded source review with local validation, not an independent +cryptographic assessment or approval to deploy the new protocol in production. + +## Scope and organization + +The audit followed the packed Hamming path from key generation and index +packing through public evaluation, response verification and private decoding. +It also examined native ownership and buffer boundaries, wire/import limits, +session state, Nitro evidence validation, deployment tooling, packaging and CI. +Existing Paillier/GPU work is already in the staging target and is preserved. + +| Layer | Location under `src/xtrace_sdk/x_vec/crypto/` | Responsibility | +| --- | --- | --- | +| BFV scheme | `encryption/bfv.py` | GMP reference arithmetic, keys, encryption, batching and ciphertext operations | +| Client | `bfv_client.py` | Existing client-style API, packed index/query layout and distance decoding | +| Public evaluator adapters | `encryption/bfv_evaluator.py`, `encryption/bfv_rns.py`, `encryption/bfv_native.py` | Reference, cached GMP, RNS, complete native and persistent-residue backends | +| Public native kernels | `bfv_cpu_ext/` | RNS/NTT, exact scaling, complete search and packed wire handling | +| Private arithmetic | `encryption/bfv_private.py`, `bfv_cpu_ext/private_decoder.h` | Optional fixed-work terminal decoder, separate from public evaluation | +| Protocol baselines | `bfv_security.py`, `bfv_verified_client.py`, `bfv_guarded_client.py` | Execution policy, bounded sessions, private result checks and independent recomputation receipts | +| Attested protocol | `bfv_attested_client.py`, `bfv_nitro.py` | Owner authorization, AWS evidence, enrollment and verification before private work | +| Parameter review | `bfv_assurance.py` | Fixed review profile and conservative circuit correctness bounds | + +The original Microsoft SEAL/TenSEAL example remains in `experiments/bfv/`. +Native BFV does not call SEAL. The evaluator backends remain independently +selectable so that the simpler implementations serve as differential oracles. +The Nitro service and relay stay in `experiments/bfv_nitro/`; production HTTP +endpoints and the existing execution context do not select BFV automatically. + +## Findings and fixes + +| Finding | Effect | Resolution and evidence | +| --- | --- | --- | +| CI's pull-request filter omitted `staging` | This PR would not run the quality gates intended for review | Added `staging` to the target filter; added offline Nitro tests, the preserved SEAL comparison and a strict documentation build | +| Malformed X.509 versions and duplicate extensions escaped the protocol error boundary | Two certificate parser exceptions bypassed the caller's expected rejection path; no decryption-gate bypass was observed | Catch `InvalidVersion` and `DuplicateExtension` explicitly; two tests mutate certificate DER and now assert `BFVProtocolError` | +| The Nitro dependency set changed PyCryptodome's loader during ASan tests | CFFI selected `RTLD_DEEPBIND`, causing ASan to abort before the binding suite could run | Set PyCryptodome's supported `PYCRYPTODOME_DISABLE_DEEPBIND=1` override before SDK import in the sanitizer harness only; rerun passed | +| Protocol message lengths were repeated in service and transport code | Wire changes could leave bounds inconsistent | Share named hello/query/receipt constants; format the new Nitro Python code consistently | +| A research link failed strict Sphinx builds, and the changelog lagged the implementation | Review documentation and ABI instructions were inaccurate | Fix the link, document public ABI 4/private ABI 1, and describe residue arithmetic, protocol baselines and Nitro status | +| The initial build record omitted native header hashes | The provenance record did not identify all arithmetic build inputs | Rebuild the audited image/EIF and record all included SDK source files, headers, build recipes and dependency locks; preserve the earlier record separately | + +The certificate regressions failed before the catch was corrected. The loader +problem was reproduced with the Nitro extra installed; the fix leaves ASan +interposition enabled and changes only the test process. No baseline BFV +arithmetic or Paillier implementation was changed by these audit fixes. + +## Correctness and attack coverage + +The tests compare decoded distances with plaintext Hamming distance and compare +the native backends with the reference arithmetic. They cover packed boundaries, +partial tiles, modulus compaction, malformed native buffers, ownership and +lifetime handling, import budgets and fixed execution policies. The SEAL +reference is exercised with synthetic inputs; it is not a security-equivalence +claim for the homemade parameters. + +The historical regression shows why result checks alone are insufficient: +a malicious evaluator can choose ciphertexts whose secret-dependent rounding +changes the owner's observable acceptance behavior. The attacker is allowed to +know the transport MAC key. Checks performed after private decoding therefore +do not close the oracle. This is the repository's protocol regression, not a +claim of a newly discovered flaw in Microsoft SEAL. SEAL also documents the +need to protect secret-dependent outputs at the application boundary in its +[security guidance](https://github.com/microsoft/SEAL/blob/main/SECURITY.md). + +The retained guarded baseline requires an independent owner-controlled verifier +to recompute the search and issue a receipt before decryption. The Nitro path +moves that trust to an approved enclave: it authorizes the owner's query, +performs one search and signs the bytes it produced. The owner verifies a fresh +attested receipt identity and then checks each receipt's session, pending query +and response binding before parsing or privately decoding the result. Tests +instrument both private backends to ensure forged receipts and the original +rounding probes never reach private work. + +That protection assumes the owner approves the correct measured program and +the enclave protects its signing key. Attestation authenticates execution under +the hardware/provider assumptions; it is not a mathematical execution proof or +a general chosen-ciphertext security transformation for BFV. See the +[Nitro protocol report](native-bfv-nitro.md) for the exact trust boundary, +owner signatures, replay handling and lifecycle limits. + +## Validation completed + +| Check | Result | +| --- | --- | +| Offline repository tests plus SEAL example | **433 passed, 1 skipped**, 102.91 s; only the real AWS enclave test skipped | +| ASan/UBSan Python binding suite | **211 passed, 1 skipped**, 45.50 s; ASan interposes `mlock`, so its failure-injection test runs only in the ordinary suite | +| Standalone native public/private arithmetic under ASan/UBSan | Passed | +| GCC and Clang `-O3` private-kernel secret-taint tests | Both clean; both deliberately leaking negative controls detected | +| Ruff and mypy | Passed; mypy checked all 50 SDK source files | +| Strict Sphinx HTML build | Passed with warnings treated as errors; external intersphinx fetching disabled for the local offline check | +| Wheel and source distribution | Built successfully; wheel includes BFV/Nitro code and native sources with the expected platform tag and optional extra | +| Audited evaluator container and amd64 EIF | Built; independent `describe-eif` matched measurements and passed CRC validation | + +The broad offline test command was: + +```bash +.venv/bin/python -m pytest tests experiments/bfv/test_packed_hamming.py \ + --ignore=tests/x_vec/test_meta_search.py \ + --ignore=tests/x_vec/test_hamming_search.py \ + --ignore=tests/x_vec/test_embedding.py -q +``` + +The ignored modules need live XTrace services or an external embedding model. +The environment had native extensions and the Nitro/SEAL optional dependencies +installed. Native sanitizer and compiler-taint commands are retained in +`.github/workflows/ci.yml`. Passing the narrow taint harness does not establish +that the entire Python client is constant time. + +## Audit benchmark and build provenance + +`benchmarks/results/native_bfv_nitro_audit_1024.json` records a new local run +after the SDK fixes. It uses the same ciphertexts in both paths, 1,024 vectors +of 512 bits, N=16,384 and the unchanged review profile. All distances were +correct and the ciphertext responses matched. The run alternates execution +order and separates one cold trial from three warm trials. + +| Operation | Warm median | +| --- | ---: | +| Existing public search | 2.432182 s | +| Search with owner authorization and response receipt | 2.436256 s | +| Owner query encryption and signing | 206.385 ms | +| Owner receipt verification, decryption and distance checks | 28.232 ms | + +This uses a test CA with actual certificate/signature verification. It measures +no NSM, vsock, enclave isolation or EC2 latency. The roughly 4 ms difference is +too small to establish a precise overhead from three warm samples. The useful +structural change remains eliminating the independent verifier's second full +search; the earlier guarded review-profile run took approximately 4.874 s +across server evaluation and recomputation. + +The response is 205,018 bytes. Query, response and receipt total 942,798 bytes, +excluding 26 bytes of RPC framing and lower transport layers. The 198-byte +receipt adds no BFV ciphertext expansion. Public setup is 194,804,883 bytes; +registration adds 141 bytes. Combined peak process RSS is 2,320,364 KiB and +includes the owner and two evaluator instances, so it is not enclave memory. +The initial local benchmark is preserved in `native_bfv_nitro_local_1024.json`. + +The audited `experiments/bfv_nitro/build-record.json` describes a 300,336,566-byte +version-4 amd64 EIF built with Nitro CLI 1.4.5, with SHA-256 +`25ddd36f47d70bb9a4ee1786fd868dfa5f8c52b8bef1981c5715bb172141c278`. +The earlier build record is preserved as `build-record-before-audit.json` and +has different measurements. Owners must approve the measurements of their +actual build; historical pins do not authorize future rebuilds. Generated EIFs +remain outside git. No real Nitro boot or fresh NSM evidence was obtained. + +## Remaining production work + +Before production use, run the opt-in test against a fresh real enclave and +exercise restart, renewal, failed enrollment, resource limits and concurrency +on the intended EC2 configuration. Measure cloud latency and enclave memory +separately from the local benchmark. Obtain independent protocol/native-parser +review and BFV parameter/private-side-channel assurance. The current correctness +bound does not certify a security level. + +The service supports one owner and immutable index per instance. Deployment +needs an admission policy, a durable owner epoch to resist configuration +rollback, and a reviewed process for withdrawing measurements and responding to +platform advisories. Certificate validity is enforced; automated revocation +retrieval is not implemented. The trusted image includes CPython and SDK +dependencies, and the host can still deny service or observe traffic metadata. +Verification failures must not trigger a fallback to unauthenticated private +decryption. These are explicit staging review items, not claims resolved by +the local code audit. diff --git a/docs/research/native-bfv-fused-rns.md b/docs/research/native-bfv-fused-rns.md new file mode 100644 index 0000000..b6da2d7 --- /dev/null +++ b/docs/research/native-bfv-fused-rns.md @@ -0,0 +1,132 @@ +# Fused NTT and exact RNS tensor scaling + +The `residue` server now uses fused negacyclic transforms and exact RNS +scale-and-round. Existing product-Q keys and ciphertexts are compatible; the +arithmetic returns the same ciphertext coefficients as the previous kernels. +Rebuild the extension for **ABI 4**. The other backends and the Paillier and +SEAL examples remain available. + +## Changes + +`bfv_cpu_ext/rns_ntt.h` folds the negacyclic twist into stage roots and folds +inverse normalization into the last butterfly. It preserves the existing +bit-reversed spectrum, uses bounded lazy residues, and unrolls groups of four +butterflies. The implementation follows the transform organization discussed by +[Longa and Naehrig](https://eprint.iacr.org/2016/504.pdf) and inspected in +[SEAL 4.1.2's transform handler](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/dwthandler.h). +No SEAL code is linked or called. + +`bfv_cpu_ext/rns_scale.h` adds an independent exact RNS conversion plan. It +removes GMP reconstruction and division from the tensor scaling loop while +preserving our reference's signed rounding. This is related to the +[textbook's RNS BFV multiplication construction](https://fhetextbook.github.io/ApplyingRNSTechniquestoFHEOperations.html), +but retains our existing binary gadget and auxiliary-prime selection. + +For each signed integer tensor coefficient z, write `z = Q*k + r`, with +`0 <= r < Q`. The desired result is exactly +`t*k + floor(t*r/Q + 1/2) mod Q`. The Q residues recover r. In an auxiliary +base B, compute `k = (z-r)/Q mod B`, then lift k with centered base conversion. +The constructor checks `B > 2*(floor(2*N*(Q-1)^2/Q)+1)`, so the quotient's sign +and value are unambiguous for every permitted tensor coefficient. + +Base conversion uses `u_j = r_j*(M/p_j)^-1 mod p_j` and +`theta = sum(u_j/p_j)`. Integer fixed-point intervals certify the floor or +nearest-integer quotient. An interval that crosses a boundary invokes an exact +18-word integer comparison. Scaling splits each `t*u_j/p_j` into its integer +part and a bounded fraction before summing; increasing t does not amplify an +approximation error. There is no floating-point arithmetic. GMP is used for +public plan constants, incoming coefficients and final response conversion. + +## Reproduce + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +.venv/bin/python benchmarks/bfv_server.py --num-vectors 1024 \ + --rns-modulus --backends residue --residue-levels 0,1,2 --seal \ + --repeats 4 --native-profile --json-out /tmp/bfv-fused-1024.json +``` + +Level 0 retains the previous kernels, level 1 adds the fused transform, and +level 2 also adds RNS scaling. Level 2 is the `residue` default. The controls +are available through `BFVEvaluator(..., kernel_level=...)` for experiments; +they do not change serialized cryptographic parameters. All three receive the +same keys, encrypted query and index. Arithmetic context creation is now +reported separately under setup; cold search still includes lazy evaluation +key and mask preparation. This changes the cold measurement boundary relative +to older reports, but leaves the warm measurement boundary unchanged. + +## Results + +AMD Ryzen 7 5800X, GCC 13.3.0 `-O3`, Python 3.12.3, GMP 6.3.0, +TenSEAL 0.3.16. One CPU evaluation thread, N=8192, t=65537, 180-bit product Q, +30-bit gadget, eta=21, 50-bit response compaction. No simultaneous tests or +other benchmark jobs. Three warm samples follow one cold search; ordering +alternates. Profiles are separate additional searches. + +| 1,024 vectors × 512 dimensions | First search (s) | Warm median (s) | +| --- | ---: | ---: | +| Previous residue kernels | 3.1222 | 2.7943 | +| Fused NTT only | 2.9633 | 2.6455 | +| Fused NTT + exact RNS scaling | 2.7570 | 2.4064 | +| Preserved SEAL prototype | 2.4590 | 2.4789 | + +The complete change takes **13.9% less warm server time**, a **1.16x speedup**. +It is 2.9% faster than this SEAL prototype in this particular run; this small +gap is not a general claim of outperforming SEAL. SEAL uses different modulus +primes, key switching and a TC128 configuration. Our parameters have no +established equivalent security claim. The native timer excludes outer +MessagePack; SEAL includes its envelope and temporary-file binding I/O. Both +include ciphertext decoding, arithmetic, compaction and encoding, and exclude +key import, private encryption/decryption and network time. + +For 32 vectors, warm medians are 0.08718 s (old), 0.08284 s (NTT only), +0.07628 s (both) and 0.07702 s (SEAL): 12.5% less time than the old kernels. +Cold times are 0.40162, 0.39037, 0.38269 and 0.08032 s respectively. Preparing +our public evaluation-key cache still dominates the small cold search. + +All native output ciphertexts match exactly, and every backend returns all +1,024 correct distances. The response is 102,496 bytes; query plus response is +471,250 bytes. This optimization does not change the wire format or noise; +the observed minimum response noise budget is 26 bits, a correctness diagnostic +and not a cryptographic security level. + +| Exclusive profile phase | Previous kernels (s) | Both optimizations (s) | +| --- | ---: | ---: | +| Forward + inverse NTT | 1.6556 | 1.5044 | +| Conversion to residues | 0.1385 | 0.0841 | +| GMP CRT reconstruction | 0.1828 (194 calls) | 0.0005 (2 calls) | +| GMP scale-and-round | 0.1678 (193 calls) | 0.0014 (1 call) | +| Exact RNS scale-and-round | 0 | 0.2152 (192 calls) | + +The 192 tensor reconstructions disappear; only the two final response +components are reconstructed. Transform-table payload falls from 2,752,512 +to 1,835,232 bytes. Key-cache payload stays 42,467,328 bytes, and the RNS scaling +constants add 1,824 bytes to the server plan. These are payload counts, not +peak process memory. NTTs still dominate the remaining cost. + +Raw samples, complete phase profiles, exact parameters, environment and source/ +binary hashes are in `benchmarks/results/native_bfv_fused_1024.json` and +`benchmarks/results/native_bfv_fused_32.json`. Fresh encryption changes key +JSON sizes slightly. Old benchmark artifacts are retained. + +After the [security hardening](native-bfv-security.md), a fresh interleaved +1,024-vector run measured 2.7874 s (previous kernels), 2.4236 s (both +optimizations) and 2.4850 s (SEAL) as warm medians. The improvement remains +**13.1%**, with identical native ciphertexts and all distances correct. +`benchmarks/results/native_bfv_security_1024.json` records that final source +state and all samples. Public-key import, measured separately, took about +1.48 s with the stricter parser and the default ~86 MB bundle. + +## Validation + +The 118 BFV test cases pass, including exact ciphertext equality across the +three kernel levels for 60-, 180- and 480-bit Q. Standalone C++ oracles compare +the fused spectrum with the old transform through N=32768, and compare exact +scaling with GMP at all eight supported Q widths. Cases cover signed convolution +bounds, zero, maximal coefficients, half-integer rounding, large t and exact +CRT fallback decisions. The C++ suite and 33 native Python-binding tests also +pass AddressSanitizer and UndefinedBehaviorSanitizer. Lint and type checks pass. + +These checks provide arithmetic and memory-safety evidence for the exercised +paths. They do not establish a security level, negligible decryption-failure +probability or production readiness. diff --git a/docs/research/native-bfv-nitro.md b/docs/research/native-bfv-nitro.md new file mode 100644 index 0000000..9834dcb --- /dev/null +++ b/docs/research/native-bfv-nitro.md @@ -0,0 +1,297 @@ +# BFV evaluation with AWS Nitro attestation + +Date: 2026-09-12 (local), with build and benchmark artifacts recorded in UTC. +Branch: `research/bfv-packed-hamming`. Baseline: `46b637b`. +Follow-up: [2026-09-15 code audit](native-bfv-code-audit.md), including updated +measurements, an audited image rebuild and additional regression coverage. + +This implements the AWS path from the +[verification plan](native-bfv-verification-plan.md). The public BFV evaluator +runs once inside an enclave and signs its own response. The owner verifies +attestation at enrollment and a receipt before each private decryption. There +is no additional owner-side BFV evaluation. The arithmetic, packing, private +decoder and post-decryption checks reuse the existing code. + +The implementation is optional and experimental. It preserves Paillier, the +SEAL example, raw BFV and the owner-controlled recomputing verifier. Production +HTTP integration is unchanged. Local validation and an EIF build are evidence +about the implementation, not a successful hardware deployment or independent +security review. + +## Code map + +| Location | New behavior | +| --- | --- | +| `src/xtrace_sdk/x_vec/crypto/bfv_attested_client.py` | `BFVAttestedClient`, `BFVAttestedServer`, signed owner requests, enrollment, response gate and protected state | +| `src/xtrace_sdk/x_vec/crypto/bfv_nitro.py` | `NitroAttestationPolicy`, AWS evidence verification, official NSM C adapter | +| `experiments/bfv_nitro/` | Measured service, untrusted relay, framed transport, owner demo, Docker/EIF build recipes and deployment README | +| `tests/x_vec/test_bfv_nitro.py` | Certificate, COSE, PCR, freshness and NSM ABI regressions | +| `tests/x_vec/test_bfv_attested_client.py` | Owner/session bindings, receipt gate, oracle regression and lifecycle tests | +| `tests/x_vec/test_bfv_nitro_transport.py` | Real framed-handler round trips, malicious framing and CLI measurement compatibility | +| `tests/x_vec/test_bfv_nitro_aws.py` | Explicitly enabled test against a fresh real enclave | +| `benchmarks/bfv_nitro.py` | Identical-input comparison, local protocol mode and AWS RPC mode | + +Install with `uv sync --extra bfv-nitro`; ordinary BFV/Paillier installations do +not acquire the optional attestation dependencies. The runbook is +`experiments/bfv_nitro/README.md`. Large EIF artifacts stay in its ignored +`artifacts/` directory. + +## Threat model and the decryption gate + +The attacker controls the parent server, relay and storage. It sees public BFV +keys, encrypted index/query/response bytes and the old transport MAC key. It can +alter, omit, reorder and replay messages, and observe behavior after the client +receives a result. The owner controls key generation, plaintext queries, the +expected encrypted setup, application code, approved measurements and the local +clock. The BFV secret and an independent owner request-signing secret stay on +the owner. + +The historical attack used crafted BFV ciphertexts whose secret-dependent +rounding could be detected through accept/reject behavior. Checks performed +after decryption did not stop that leakage. This protocol therefore requires +evidence of approved evaluation **before** ciphertext parsing or private +decoding. It authenticates the full packed-distance computation, not merely a +server-supplied output hash. + +Under the additional assumption that approved enclave code executes correctly +and its signing secret remains protected, a malicious parent cannot sign a +replacement ciphertext. Nor can it obtain a receipt for a chosen malicious +query: the enclave requires a separate owner signature, and the client accepts +only receipts matching its own pending queries. The existing transport MAC +does not supply either authority. + +AWS Nitro supplies isolation and attestation. The application must still +validate evidence and approve the measured program. +[AWS Nitro environment](https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave.html). +If that program or the TEE is compromised, receipt-key misuse can reopen the +decryption oracle. This is not a chosen-ciphertext security transformation or a +mathematical proof of correctness for raw BFV. + +```text +Owner Untrusted EC2 parent Nitro enclave +BFV secret, owner signing key ciphertexts and relay public BFV setup +approved PCRs and expected epoch NSM receipt key + | signed public registration -----------------------------> | + | signed nonce + context ---------------------------------> | + | <----- AWS document binds PCRs, nonce, context, receipt key | + | <------------------------------ proof of key possession -- | + | verify evidence and store a short-lived receipt identity | + | signed encrypted query ---------------------------------> | + | evaluate ONCE | + | <----------------------- encrypted distances + signature -- | + | verify session, pending query, response hash and signature | + | decrypt, check distances, select top three locally | +``` + +## Attestation verification + +`bfv_nitro.py` uses a deliberately narrow Nitro document profile. OpenSSL checks +the certificate path; `cryptography` checks the P-384/ES384 COSE signature; +`cbor2` provides bounded decoding. The new code implements the format limits +and application policy. This integration has not received independent review. + +- The only production trust anchor is the SHA-256 fingerprint of the AWS root + DER certificate: `641a0321a3e244efe456463195d606317ed7cdcc3c1756e09893f3c68f79bb5b`. + A root sent by the peer is untrusted until that fingerprint matches. +- Validate the complete path, validity periods, critical constraints, CA + authorization and key usage. Accept no unused or duplicate certificates. + Real Nitro leaf certificates omit AKI/SKI, so OpenSSL's strict-profile flag + would incorrectly reject them; normal path validation plus explicit + application usage checks handles the actual AWS profile. +- Require protected algorithm ES384, a 96-byte signature, SHA384 PCRs and + bounded claims. Reject duplicate CBOR keys, indefinite items, semantic tags, + excessive nesting and trailing data. The optional outer COSE tag 18 is + accepted. Documents are limited to 16 KiB. +- Require exact nonzero owner-provisioned PCR0, PCR1 and PCR2. Optional other + PCRs are additional requirements. Debug measurements and peer-provided + replacement pins are never acceptable. +- Require the current owner's 32-byte nonce and context, plus a canonical + Ed25519 SubjectPublicKeyInfo. Reject small-order identities. Verify a + separate proof of possession of that receipt key. +- Check local evidence age, future clock tolerance and handshake timeout. + Receipt acceptance expires after at most five minutes and also respects + certificate expiry. Monotonic time limits the lease. + +AWS specifies the evidence format, root and PCR meanings in its +[validation guide](https://docs.aws.amazon.com/enclaves/latest/user/verify-root.html) +and [measurement guide](https://docs.aws.amazon.com/enclaves/latest/user/set-up-attestation.html). +Exact image pins remain required if an EIF is signed: PCR8 alone could authorize +other programs signed by the same image-signing key. + +The server calls AWS's official NSM library for entropy and evidence. The image +builds version 0.5.2 from commit +`1993eeb0620d35f5cefc50b17638b432325328f9`, with a checked archive hash and a +local Cargo lock. The SDK adapter calls its documented C ABI via `ctypes`. +[Official NSM source](https://github.com/aws/aws-nitro-enclaves-nsm-api). +There is no random-seed fallback if the NSM device fails, and no mock provider +in the deployed service entrypoint. + +## Versioned bindings and wire format + +All hashes below are SHA-256. Concatenated fields have fixed widths where +specified. Existing inner BFV records and the registration use bounded +MessagePack; epochs are unsigned 64-bit integers. Ed25519 uses separate RFC8032 +contexts for owner authorization, enrollment possession and response receipts. + +The application context commits to the shared circuit identifier, sorted JSON +execution-policy digest, **actual serialized setup** digest, 32-byte owner +public key and big-endian epoch. The setup includes the ordered IDs, count, +encrypted index, public/evaluation keys and parameters. The policy commitment +includes resource limits as well as arithmetic. Context and session hashes +have separate versioned domain prefixes in the code. + +| Message | Binding and format | +| --- | --- | +| Registration | MessagePack `[setup, transport_mac_key, owner_public_key, epoch, owner_signature]`; signature covers `XBNI01`, context and transport-key hash | +| Enrollment challenge | `XBNH01 || context[32] || nonce[32] || owner_signature[64]` (134 bytes) | +| Evidence | AWS COSE document with `nonce`, `user_data=context`, and the enclave's DER receipt public key | +| Possession proof | Ed25519 enrollment signature over the unsigned challenge and document hash (64 bytes) | +| Session ID | Domain-separated hash of context, enrollment nonce and raw receipt public key | +| Query | `XBNQ01 || session[32] || inner_query || owner_signature[64]`; signature also binds context and inner-query hash | +| Receipt | `XBNR01 || context[32] || session[32] || query_hash[32] || response_hash[32] || signature[64]` (198 bytes) | + +The server verifies the owner's registration signature before BFV key JSON +import and hashes the bytes it actually imports. Once registered, the public +setup cannot be replaced through the service. Each search verifies owner +authorization before inner parsing, performs exactly one complete evaluation, +then signs the produced response. It cannot sign an externally supplied +response, cache entry or digest. + +The client checks bounds, receipt bindings, its locally pending request digest +and the enclave signature before calling the existing inner client's +`finish_query`. The outer lock spans this gate and private work so renewal +cannot race with decryption. The inner ticket is consumed once. A valid receipt +for an unrelated or already completed request is insufficient. + +The transport adds a nine-byte header, `XBN1 || kind:u8 || length:u32be`, to each +RPC leg. It validates operation and length before reading the body, imposes an +absolute read deadline and handles one RPC per connection. A search reply is a +MessagePack `[response, receipt]`. At the measured 1,024-vector size, framing +adds 26 bytes beyond the query, response and receipt payload totals; TCP, vsock +and tunnel overhead are separate. The relay has four bounded workers. The +enclave services one request at a time and exposes no diagnostic payload logs. + +## Lifecycle and deployment limits + +This first service is single-owner and single-index. Initial registration is +self-authorized; whoever reaches an empty instance first can claim it. The +legitimate client will reject evidence for a different owner/setup. Account +admission and host availability remain deployment concerns. + +Renewing attestation replaces the active session and discards pending queries +without importing the index again. Nonces are single-use and retained in a +bounded set of 1,024 entries. Restarting the enclave discards its receipt key, +replay state and imported index. A restart requires re-upload and fresh +attestation; an index update requires a new service instance and owner epoch. + +Protected client exports include the owner signing secret and existing BFV +private state under a separate AES-GCM wrapping key. The expected setup, epoch +and both policies authenticate the envelope. They exclude enrollment and +pending queries; restore always requires fresh attestation. An owner must +retain its expected epoch in trusted durable state: restoring an entire old +owner configuration is outside what an encrypted backup can detect. + +The entrypoint fixes the larger-ring review profile and public residue backend. +No host configuration selects weaker parameters, different libraries or a GPU +outside the enclave. BFV decryption remains on the owner, and selection of the +closest results remains local. This change does not add homomorphic top-k. + +## Initial local measurements and image build + +`benchmarks/results/native_bfv_nitro_local_1024.json` records the initial local +run on a Ryzen 7 5800X with 1,024 vectors of 512 bits, N=16,384 and the unchanged +review profile. Both paths evaluated the **same ciphertexts** and returned +identical response bytes and correct plaintext distances. Measurements use one +cold trial and three warm trials, with the execution order alternated. + +| Measurement | Warm median | Interpretation | +| --- | ---: | --- | +| Existing public evaluation | 2.438622 s | One baseline search | +| New evaluation plus owner/receipt protocol | 2.439097 s | One search with signatures and session checks; synthetic attestor, no hardware | +| Owner query encryption and signing | 203.155 ms | Full query creation, not signature time alone | +| Owner receipt verification and finish | 28.105 ms | Includes parsing, private decoding and distance checks | +| Synthetic evidence verification | 4.992 ms | Test CA certificate path, COSE and possession checks; not AWS NSM latency | + +The observed search difference is about 0.5 ms, below what these few samples +can resolve reliably. There is no meaningful speedup or precise overhead claim. +The structural improvement is avoiding the second complete search: the +historical guarded baseline took approximately 4.874 s for server evaluation +plus owner recomputation. Actual enclave overhead and cloud tail latency remain +unmeasured. + +| Payload | Bytes for this run | Change from the same inner BFV payload | +| --- | ---: | --- | +| Public setup | 194,805,676 | Unchanged BFV setup format | +| Signed registration | 194,805,817 | Adds 141 bytes once | +| Query | 737,582 | Adds 102 bytes per request | +| Response | 205,018 | Identical ciphertext bytes | +| Receipt | 198 | Same length as the existing guarded receipt; distinct protocol | +| Query + response + receipt | 942,798 | Excludes 26 bytes of RPC framing and lower transport layers | + +The setup includes public/evaluation keys and encrypted index, not private +BFV state. Its JSON integer lengths vary slightly with fresh random keys. The +synthetic document was 1,938 bytes plus a 64-byte possession proof; that is not +the real AWS document size. Process peak RSS was 2,319,872 KiB and includes the +owner, two evaluators and serialization buffers. It is not enclave-only memory. + +The initial evaluator container and an **amd64 EIF** were built locally using the +official Nitro CLI 1.4.5. An independent `describe-eif` invocation reproduced +PCR0/1/2 and passed the EIF CRC check. That EIF was 300,334,519 bytes, version 4, +and has SHA-256 +`a6b354a039abdda7399768a6d9b9138f729128d47ad60830dc33b2c074bcd70c`. +It is unsigned as an image artifact; the client pins its exact measurements +and still requires a valid **AWS-signed attestation document** at runtime. +Image signing and runtime attestation are different operations. + +`experiments/bfv_nitro/build-record-before-audit.json` preserves that initial +build's PCRs and metadata. The current `build-record.json` and ignored +`artifacts/` output describe the audited rebuild documented in the +[code audit](native-bfv-code-audit.md). Its source hashes include the native +headers, and its measurements differ from the initial build. A trusted local +build record is not proof that the program has booted in an enclave. No EC2 +resources were created and no fresh NSM evidence was obtained here. + +## Validation and remaining release work + +Initial local checks: **346 BFV tests passed, one real-AWS test skipped** in +50.30 seconds. The focused Nitro set contains 77 passing tests. Ruff passed on +the SDK and all new Python files, and mypy passed on all 50 SDK source files. +The lockfile check passed offline. Sphinx built successfully with the existing +unrelated missing-document warning in `bfv-packed-hamming.md:259`. The container +and EIF build checks described above also passed. No baseline arithmetic source +or the `main` branch was modified. + +The subsequent [code audit](native-bfv-code-audit.md) fixed the documentation +warning and certificate error handling, added staging PR coverage, and passed +433 offline repository/reference tests plus 211 sanitizer-backed binding tests. +That report records the precise scope and skips for each run. + +The offline suite covers exact output equality with the existing evaluator and +an assertion that only one evaluation occurs. Both private decoding backends +are instrumented during forged-receipt and historical rounding-oracle probes +to ensure that rejection happens before private work. Additional tests cover +wrong roots and certificate constraints, wrong/zero PCRs, stale evidence, +nonce/context/key substitution, failed possession, forged owner requests, +unrelated valid receipts, replay, renewal, expiry, restore binding and +concurrent completion. Socket tests exercise the actual framed service. + +A public historical AWS document is accepted at its original valid time and +rejected at today's clock. It is not fresh evidence for this application. A +test-only C shim checks NSM argument ordering, lengths, failure statuses and +resource cleanup; the built official library loads and fails closed when the +real NSM device is absent. Synthetic roots exist only in test/benchmark code; +the production verifier rejects them by default. + +The next release work is actual Nitro EC2 end-to-end execution and lifecycle +testing, matched cloud performance and isolated memory measurements, independent +protocol review, native/parser attack-surface review, and the existing BFV +parameter and client private-key side-channel assurance. The measured software +includes CPython, the SDK import tree and all image dependencies; shrinking that +trusted footprint is useful follow-up work. + +Certificate expiry is enforced, but this verifier does not fetch CRLs or provide +an automated platform-advisory/revocation service. Owners need a reviewed process +for withdrawing measurements and updating trusted builds. Attestation establishes +program identity under AWS's assumptions; it does not remove these operational +and implementation obligations. Host denial of service and metadata observation +remain possible. Cryptographic execution proofs remain a separate future track. diff --git a/docs/research/native-bfv-ntt.md b/docs/research/native-bfv-ntt.md new file mode 100644 index 0000000..b22244e --- /dev/null +++ b/docs/research/native-bfv-ntt.md @@ -0,0 +1,232 @@ +# Native BFV RNS/NTT server arithmetic + +The optional `server_backend="rns"` evaluator uses independently implemented C++ +kernels to accelerate native BFV. It retains the existing keys, ciphertext +modulus, error distribution, packing layout and wire format. The default +`"optimized"` GMP evaluator and the original `"reference"` evaluator remain +available. The earlier SEAL experiment is preserved for comparison. + +The subsequent [`native` server backend](native-bfv-server.md) runs complete +searches inside C++, adds native phase profiling, and further optimizes this +arithmetic while preserving the same keys and ciphertexts. + +## What we learned from SEAL + +The source review used the pinned **SEAL v4.1.2** tag, rather than an unversioned +branch. The comparison executable is the separately installed TenSEAL prototype; +its package version and actual modulus sizes are recorded by the benchmark. + +| SEAL source | Relevant technique | Native implementation | +| --- | --- | --- | +| [`evaluator.cpp`, `bfv_multiply`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/evaluator.cpp) | RNS base extension, NTT polynomial products, and scale/base conversion | Auxiliary RNS bases reconstruct the exact signed tensor product before our existing rounding rule | +| [`keygenerator.cpp`, `generate_one_kswitch_key`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/keygenerator.cpp) | Evaluation keys are generated in NTT form | Lazily convert our existing gadget keys into NTT form and cache them | +| [`evaluator.cpp`, `switch_key_inplace`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/evaluator.cpp) | Pointwise key products and delayed reduction in wide accumulators | Sum gadget products in unsigned 128-bit accumulators before inverse transforms | +| [`util/ntt.cpp`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/ntt.cpp) | Prepared transform tables and inverse scaling | Cached roots, twists and inverse scaling; matching forward/inverse orders avoid explicit bit reversal | +| [`util/uintarithsmallmod.h`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/uintarithsmallmod.h) | Precomputed quotient operands for modular multiplication | Shoup multiplication avoids division in transform butterflies | + +SEAL's multiplication also uses specialized RNS conversion routines in +[`util/rns.cpp`](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/rns.cpp). +Our backend deliberately keeps the original BFV scheme and exact GMP +reconstruction, so it does not implement SEAL's full BEHZ multiplication or +special-prime key-switching scheme. No SEAL source is compiled or linked into +the native extension. These are independent implementations of the arithmetic +techniques identified during that review. + +## Arithmetic and correctness bounds + +The existing q is a single large prime and need not support an NTT of the +required length. The backend instead selects distinct 60-bit auxiliary primes +`p_i = 1 mod 2N`. A polynomial is temporarily represented by its residues modulo +each `p_i`. These primes are arithmetic scratch space; they do not replace q. + +Each prime gets an N-point negacyclic transform. Multiplying coefficient i by +`psi^i`, where psi has order 2N, changes multiplication modulo `X^N+1` into a +cyclic transform calculation. The forward transform produces bit-reversed +values; the complementary inverse accepts that same order and combines division +by N with the inverse twist. Pointwise products compute the convolution. + +The Chinese Remainder Theorem reconstructs the **signed integer** result. Let +`M = product(p_i)`, b be the gadget digit width and L the number of digits. +We choose enough primes that: + +- Key switching: `M > 2*N*L*(2^b - 1)*(q - 1)`. +- Ciphertext multiplication, including the two cross products: + `M > 4*N*(q - 1)^2`. +- A general polynomial product uses the bound + `M > 2*N*max(lhs)*max(rhs)`. + +Every result coefficient then lies strictly between `-M/2` and `M/2`, making +centered CRT reconstruction unique. For BFV multiplication, the reconstructed +coefficient c is rounded with `floor((2*t*c + q)/(2*q))` before reduction modulo +q. Premature reduction modulo q would destroy information needed by that step. +The default key switch uses four auxiliary primes; the tensor product uses +seven. All transforms and rounding use integer arithmetic. + +Transform values remain canonical residues. A fixed twiddle w stores +`floor(w*2^64/p)`; its high-word product gives a quotient estimate requiring at +most one correction. Gadget products are below `2^120`, allowing batches of +256 terms in 128-bit accumulators. Longer gadgets reduce between batches. +Digits of at most 60 bits are extracted directly from GMP limbs on supported +64-bit GMP builds; larger digits retain a GMP path. + +The complete subtract/square/relinearize/rotate/add/mask circuit for one Hamming +tile stays inside C++. This avoids returning thousands of intermediate GMP +coefficients to Python after each rotation. Result merging and public client +interfaces retain their existing structure. Native calls release the GIL during +computation; immutable shared plans and per-call scratch arrays support concurrent +evaluations. Input lengths, coefficient ranges, capsule types and native plan +ownership are checked at the extension boundary. + +## Build and use + +From the repository root, using a Linux 64-bit environment with a C++17 compiler, +Python development headers and GMP development files: + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +``` + +There is no additional runtime Python package, pybind11, SEAL, or CUDA dependency. +The built module needs the system GMP and C++ runtime libraries. It is optional; +the Python/GMP path works without a compiler or this extension. Wheels include +the native binary when built locally and receive a Python/OS/architecture tag. + +```python +import json +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient + +client = BFVClient(embed_len=512) +server = BFVClient(skip_key_gen=True, server_backend="rns") +server.load_config(json.loads(client.stringify_config())) +server.load_stringified_keys(client.stringify_pk()) +assert server.keys is None + +query = [0, 1] * 256 +vectors = [query[:], [1 - bit for bit in query]] +response = server.encode_hamming_server_packed( + client.encrypt_vec_one(query), client.encrypt_vec_packed(vectors), len(vectors) +) +assert client.decode_hamming_client_packed(response, len(vectors)) == [0, 512] +``` + +Reuse the server to amortize transformed-key preparation. Key loading discards +the cache. Select `server_backend="optimized"` for the previous GMP optimization +or `"reference"` for the original arithmetic. `"rns"` reports a build instruction +if the native extension cannot be imported; it does not silently change the +requested backend. The setting is runtime-only and does not alter saved crypto +configurations. + +## Code map + +| File | Responsibility | +| --- | --- | +| `crypto/bfv_cpu_ext/rns_ntt.h` | NTT plans, fixed-multiplier reduction, exact CRT, gadget multiplication, BFV scaling and native tile evaluation | +| `crypto/bfv_cpu_ext/bindings.cpp` | Checked CPython boundary and native object lifetimes | +| `crypto/bfv_cpu_ext/Makefile` | Optional local C++ build | +| `crypto/encryption/bfv_rns.py` | Existing GMP polynomial types to native buffers and cached key handles | +| `crypto/encryption/bfv_evaluator.py` | Backend selection and common validation/reference arithmetic | +| `crypto/bfv_client.py` | Existing individual/packed Hamming interfaces | +| `tests/x_vec/test_bfv_rns.py` | Independent arithmetic oracles, bounds, native lifetimes, concurrency and default parameters | +| `tests/x_vec/native/test_bfv_rns.cpp` | Standalone native oracles for sanitizer runs | +| `benchmarks/bfv_server.py` | Repeated native comparisons and optional SEAL comparison | + +Crypto paths above are relative to `src/xtrace_sdk/x_vec/`. + +## Verification and benchmarks + +```bash +.venv/bin/python -m pytest tests/x_vec/test_bfv_encryption.py \ + tests/x_vec/test_bfv_evaluator.py tests/x_vec/test_bfv_rns.py \ + tests/x_vec/test_bfv_client.py -q + +g++ -std=c++17 -O1 -g -fsanitize=address,undefined -fno-omit-frame-pointer \ + -I src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext \ + tests/x_vec/native/test_bfv_rns.cpp -lgmpxx -lgmp -o /tmp/test_bfv_rns +/tmp/test_bfv_rns + +.venv/bin/python benchmarks/bfv_server.py --num-vectors 1024 \ + --backends optimized,rns --seal --repeats 3 --json-out /tmp/bfv-rns-1024.json +``` + +Omit `--seal` when TenSEAL is not installed. The test suite covers zero, maximum, +sparse and random coefficients; digit boundaries at 59/60/61 bits; 512 one-bit +digits; exact signed rounding; repeated multiplication; malformed buffers; +partial tiles and multiple responses; public-only evaluation in another process; +key reload; and missing-extension fallback. Standalone tests reach N=32768. +LeakSanitizer cannot run under the local tracing sandbox: setting +`ASAN_OPTIONS=detect_leaks=0` keeps address and undefined-behavior checks enabled +there. CI runs the standalone sanitizer executable without that override. + +Native backends share identical public keys and encrypted inputs. Every timed +search must produce identical native ciphertexts and correct plaintext distances. +SEAL uses the same vectors with its own keys and parameters; its result is +checked against every plaintext distance. Search order alternates, and first-use +preparation is reported separately from subsequent warm searches. Profiles are +extra runs excluded from timing samples. + +Native timings include ciphertext decoding/encoding and terminal compaction, +but exclude the outer MessagePack envelope. SEAL timings include its envelope +and temporary-file binding I/O. Neither includes key import, index/query +encryption, client decryption, network time, or top-k content retrieval. The +comparison measures practical server calls; it does not establish equal security +or identical cryptographic work across schemes. + +Validation for this change passed 151 offline tests across the BFV experiment, +CLI and SDK crypto suites, including 69 BFV tests. API integration and GPU tests +were excluded. Ruff, formatting checks, mypy and the standalone address/undefined- +behavior sanitizer run passed. A built platform-tagged wheel loaded its own +native module and completed an encrypted Hamming search. The Sphinx build passed +with the pre-existing README cross-reference warning in `bfv-packed-hamming.md`. + +### Recorded measurements + +Measured on 2026-09-10 on an AMD Ryzen 7 5800X with Python 3.12.3, GMP 6.3.0, +GCC 13.3.0 (`-O3`, without `-march=native`) and TenSEAL 0.3.16. Each vector has +512 bits. Native parameters +are N=8192, t=65537, q=180 bits, six 30-bit gadget digits and a 50-bit terminal +response modulus. The SEAL context has four active primes of 43/43/44/44 bits, +plus a 44-bit special prime for keys, with its TC128 security setting. + +| Vectors | Server backend | First search (s) | Warm median (s) | +| ---: | --- | ---: | ---: | +| 32 | Previous GMP (`optimized`) | 1.7459 | 1.6894 | +| 32 | Native C++ RNS/NTT (`rns`) | 0.5007 | 0.2624 | +| 32 | Preserved SEAL prototype | 0.0775 | 0.0748 | +| 1,024 | Previous GMP (`optimized`) | 55.7613 | 55.9025 | +| 1,024 | Native C++ RNS/NTT (`rns`) | 8.3291 | 8.0355 | +| 1,024 | Preserved SEAL prototype | 2.4032 | 2.3901 | + +The 32-vector run has one first search and three warm searches per backend; +the 1,024-vector run has one first search and two warm searches. On the larger +workload the native backend is **6.96 times faster** than the previous GMP +backend. SEAL remains **3.36 times faster**, compared with 23.39 times faster +than GMP in the same measurement. These are local repeated measurements, not +a claim about all machines or parameter sets. + +Both native backends returned exactly the same encrypted responses on every +run. For 1,024 vectors, each response is 102,496 bytes and query plus response +is 471,250 bytes with the benchmark's MessagePack envelope. The minimum native +response noise budget is 26 bits. The arithmetic optimization changes neither +those ciphertext bytes nor the original scheme's security assumptions. + +The 1,024-vector native server caches 16 transformed keys: 50,331,648 bytes of +coefficient arrays plus 2,752,512 bytes of transform tables. The previous GMP +server reports 52,083,728 bytes of prepared-key payload. Both are additional +to the loaded public key and exclude containers, CRT constants and temporary +allocations; these figures are not peak process memory. + +Raw samples, parameters, correctness checks, source/binary hashes and timing +boundaries are saved in `benchmarks/results/native_bfv_rns_32.json` and +`benchmarks/results/native_bfv_rns_1024.json`. The additional Python profile is +`benchmarks/results/native_bfv_rns_profile_32.txt`. Profiles show native tile +evaluation alongside the remaining Python validation and serialization work; +they do not separately attribute time inside the C++ kernel. + +## Remaining differences from SEAL + +The native backend still reconstructs large GMP coefficients after key switches +and uses them between operations. SEAL has an integrated RNS representation and +specialized base conversion. Our backend also allocates scratch arrays per call +and retains Python wire parsing/validation and result merging. Those are further +optimization opportunities. Current code remains experimental, variable-time +leveled BFV with the same limitations as the [native implementation](native-bfv.md). diff --git a/docs/research/native-bfv-predecryption.md b/docs/research/native-bfv-predecryption.md new file mode 100644 index 0000000..0567abe --- /dev/null +++ b/docs/research/native-bfv-predecryption.md @@ -0,0 +1,466 @@ +# BFV approval before decryption, private arithmetic and parameter review + +Date: 2026-09-11. Branch: `research/bfv-packed-hamming`. + +**The new `BFVGuardedClient` blocks the demonstrated malicious-response oracle +under an explicit additional trust assumption:** an owner-controlled verifier +independently recomputes the prescribed public BFV result and approves its exact +bytes. The client checks that approval before parsing or decrypting the result. +The verifier needs public BFV keys and ciphertexts, but no BFV secret key, +plaintext vectors or private result summary. + +This is an implemented protocol baseline for review, not a claim of general +CCA2-secure BFV or production readiness. It costs another full evaluation and +depends on the verifier's integrity and signing-key custody. It is neither a +succinct cryptographic proof nor attestation of an untrusted server. A signer +controlled by the malicious evaluator provides no protection. If the owner can +simply use the trusted verifier as the evaluator, the extra untrusted evaluation +can be avoided at the application level; this implementation intentionally +measures independent checking of an existing evaluator. + +The [earlier session report](native-bfv-verified-sessions.md) remains the record +of the vulnerable post-decryption check. Its attack test is retained. The raw +BFV and old session APIs retain their behavior and require explicit migration +to acquire the new gate. Paillier, the SEAL experiment and production endpoints +are unchanged. This work is internal, AI-assisted engineering, with no +independent cryptographic review. + +## Code map and local use + +All paths below are relative to the repository root. + +| File | Purpose | +| --- | --- | +| `src/xtrace_sdk/x_vec/crypto/bfv_guarded_client.py` | Owner verifier, Ed25519 receipts, mandatory pre-decryption checks and bound private exports | +| `src/xtrace_sdk/x_vec/crypto/bfv_assurance.py` | Larger-ring review profile and exact conservative circuit bounds | +| `src/xtrace_sdk/x_vec/crypto/encryption/bfv_private.py` | Python adapter to the separate private native decoder | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_decoder.h` | Fixed-width private NTT, CRT, rounding, locked buffers and wiping | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/private_bindings.cpp` | Bounded CPython interface, capsule ownership and GIL handling | +| `src/xtrace_sdk/x_vec/crypto/bfv_verified_client.py` | Small change: optional native private decoding; original default remains Python | +| `tests/x_vec/test_bfv_guarded_client.py` | Forgery/oracle, pin, replay, concurrency and saved-state regressions | +| `tests/x_vec/test_bfv_private.py` | Independent arithmetic, boundary, ownership and memory-lock tests | +| `tests/x_vec/test_bfv_assurance.py` | Bound checks and observed-phase comparisons | +| `tests/x_vec/native/test_bfv_private.cpp` | Standalone sanitizer, wipe and compiled secret-taint tests | +| `benchmarks/bfv_guarded.py` | Same-ciphertext performance and all communication legs | +| `benchmarks/bfv_parameter_audit.py` | Actual parameter manifest and pinned external lattice estimates | + +Build both optional CPU extensions on Linux with GCC or Clang: + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +``` + +The public `_bfv_rns` ABI remains 4; the new, separate `_bfv_private` ABI is 1. +The guarded client defaults to native private decoding and the review profile. +Missing binaries, incompatible ABIs, unsupported parameters and memory-lock +failure raise errors; there is no automatic private-backend fallback. +`private_backend="python"` is an explicit research option with the same receipt +gate and the old variable-time private implementation. + +```python +import secrets + +from xtrace_sdk.x_vec.crypto.bfv_assurance import bfv_review_policy +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient +from xtrace_sdk.x_vec.crypto.bfv_guarded_client import ( + BFVGuardedClient, BFVPublicVerifier, bfv_verifier_public_key, +) +from xtrace_sdk.x_vec.crypto.bfv_verified_client import BFVVerifiedServer + +policy = bfv_review_policy() # N=16384; fresh keys and index required. +authentication_key = secrets.token_bytes(32) # Shared with evaluator. +signing_seed = secrets.token_bytes(32) # Owner-controlled verifier ONLY. +wrapping_key = secrets.token_bytes(32) # Private client storage ONLY. +pin = bfv_verifier_public_key(signing_seed) # Provision through a trusted path. +client = BFVGuardedClient( + BFVClient(**policy.config()), authentication_key, pin, policy=policy +) +setup = client.prepare_index( + [[0] * 512, [1] * 512], vector_ids=["document-a", "document-b"] +) + +# Untrusted evaluator; only public BFV material is imported. +server = BFVVerifiedServer(setup, authentication_key, policy=policy) + +# Run locally on the owner, or in a SEPARATELY TRUSTED owner-controlled service. +# Obtain the expected digest directly from the owner, not from the evaluator. +verifier = BFVPublicVerifier( + setup, authentication_key, signing_seed, + expected_setup_digest=client.setup_digest, policy=policy, +) +request = client.begin_query([0] * 512) +response = server.search(request) +receipt = verifier.approve(request, response) # A second full public evaluation. +distances = client.finish_query(response, receipt) +assert distances == [0, 512] + +protected = client.protect_state(wrapping_key) +restored = BFVGuardedClient.restore( + setup, protected, authentication_key, wrapping_key, pin, policy=policy +) +``` + +This example is local orchestration, not a network service or secure process +isolation. In deployment the signing seed must never reach the untrusted +evaluator. Pin both the verifier identity and owner setup through trusted +configuration, authenticate owner authorization at the verifier, and review +the actual components running inside that trusted service. + +## Why approval changes the oracle boundary + +The previous result fingerprint checked plaintext *after* decryption. A +malicious server could submit carefully chosen ciphertexts and learn a secret +coefficient from the client's acceptance. Transport authentication could not +exclude that server because it knew the shared authentication key. + +The new sequence is: + +1. The owner creates an immutable authenticated index setup and a fresh query + ticket, retaining the SHA-256 digest of its exact encrypted request. +2. The evaluator produces its response. The owner-controlled verifier uses an + independently imported copy of the owner-pinned setup to recompute the + complete canonical response, including terminal modulus switching. +3. The verifier compares every response byte, then signs only that transcript. + Its API cannot sign arbitrary caller-provided hashes. These decisions use + public ciphertexts only; even malicious queries cannot invoke BFV decryption + at the verifier. +4. The client checks fixed receipt length, version, circuit, setup, pending + owner-request binding, response digest and Ed25519 signature **before BFV + response parsing or any private decoding**. Rejection cannot expose the + BFV decryption predicate used in the historical attack. +5. The inner session still consumes the request once, enforces response shape + and modulus, and checks the private Hamming fingerprint before returning + the complete distance list. Concurrent replays cannot decrypt twice. + +The owner-request check is essential. The evaluator knows the transport MAC +key and can construct its own authenticated requests. A valid receipt for one +of those requests must not authorize decryption on the owner. Tests cover this +case as well as the earlier rounding-oracle ciphertexts and check that neither +the GMP nor native decoder is reached. + +Each receipt is **198 bytes**: a 6-byte version prefix, four 32-byte fields +(circuit identifier, setup digest, request digest, response digest), and a +64-byte Ed25519 signature. Signing uses a versioned Ed25519 context. The pin +comes only from local configuration; small-order public keys, including the +encoded identity accepted by the underlying import API, are explicitly +rejected. Cryptographic signing uses the existing dependency, not a new +signature implementation. +[PyCryptodome EdDSA documentation](https://www.pycryptodome.org/src/signature/eddsa). + +The circuit identifier describes the implemented deterministic computation; +it is not a proof that arbitrary software follows it. Semantic or canonical +wire changes require a new circuit version, compatible trusted code and fresh +deployment review. A bad proposed result consumes the verifier's query ticket, +so a retry needs a new owner request. Missing, forged or unrelated receipts do +not consume the client's legitimate pending ticket. The outer AES-GCM saved +state additionally binds this gate and verifier pin, preventing accidental +restoration through the old unguarded session API. It adds 34 bytes over the +inner protected export and does not supply durable anti-rollback protection. + +This implements an explicit trusted-computation boundary, **not** the +SNARK-based transformation analyzed in the verifiable-FHE literature. +That literature motivates binding permitted outputs to original inputs before +opening a decryption interface; it does not prove this SDK protocol secure. +[Manulis and Nguyen, EUROCRYPT 2024](https://eprint.iacr.org/2024/202). +SEAL also warns that decryptor outputs/noise information require private +handling; importing a ciphertext does not establish its authenticity. +[SEAL security guidance](https://github.com/microsoft/SEAL/blob/main/SECURITY.md). + +## Private arithmetic: scope and evidence + +The separate C++ decoder imports a ternary key once and keeps two secret NTT +spectra. For each two-component terminal ciphertext it computes exact +negacyclic `c1*s`, adds `c0`, rounds the phase modulo the plaintext modulus, +and batch-decodes all N slots. The input/output lengths and loop bounds depend +on public parameters, never a secret coefficient or decoded distance. + +Two descending 60-bit NTT primes allow exact signed CRT reconstruction: +`|c1*s| < N*q < 2^65`, while half their product exceeds `2^117`. The private +path supports power-of-two N from 8 through 32768, `t < 2^30` with batching +roots, and terminal `t < q < 2^50`. Fixed 67- and 82-iteration binary dividers +replace division on secret phase values. Root setup and Shoup precomputation +can divide public values; private multiplication uses fixed-width products +and masked correction. The public server's faster variable-time NTT is not +reused for secret transforms. + +Testing exposed Clang 18 turning ordinary masked corrections into branches. +A GCC/Clang register value barrier now prevents that optimization in the +tested builds. This is a known compiler-control technique, also documented in +[BoringSSL's constant-time helpers](https://boringssl.googlesource.com/boringssl/+/refs/heads/main/crypto/internal.h). +The resulting GCC 13.3 and Clang 18 `-O3` kernels pass dynamic secret-taint +checks for N=16,128,8192 and positive, negative and mixed ternary keys. The test +marks persistent secret spectra and transform inputs undefined in Valgrind, +then declassifies plaintext only after decoding returns. A deliberate secret +branch is a negative control and must fail. CI runs both compilers. + +These tests detect executed secret-dependent branches and addresses. They do +not prove constant latency for every instruction, compiler, processor or +input, or cover speculative execution, faults, power analysis, hostile kernels +or physical access. Source review, the independent arithmetic oracles and +sanitizers supplement the taint tests; none is a side-channel certification. + +Persistent native secret buffers and temporary private work use separate +`mmap` allocations, mandatory `mlock`, Linux `MADV_DONTDUMP` and volatile +full-region wiping. Lock/exclusion failure stops the operation. Decode and +close share a mutex; closing waits without holding Python's GIL and wipes the +native key before rejecting further decode calls. Capsule destruction also +wipes. At N=16384, a decoder locks 262,144 persistent bytes plus 393,216 +temporary bytes during a decode; page rounding may matter for small rings. +Provision sufficient per-process lock limits. An isolated OS-limit test checks +that failure never silently falls back to unlocked storage. + +**The whole private client is still not constant time or fully erasable.** +Python/GMP key generation, key import, encryption, private summary checking, +Python key objects, temporary imported key bytes and returned plaintext remain +outside the native kernel's guarantees. Existing keys are retained for future +encryption and protected exports. Wiping native buffers does not erase those +copies, registers, allocator remnants or the caller's results. A production +review must address those paths and process/key custody as well. + +## Concrete parameter estimates + +The runner records the actual product Q, not the larger multiplication +workspace basis. Both profiles use t=65537, uniform ternary secret/ephemeral +coefficients, centered-binomial error eta=21 (variance 10.5), six 30-bit gadget +digits and a 50-bit terminal response prime. + +| Profile | N | Q primes | Public-key JSON cap | +| --- | ---: | --- | ---: | +| Original arithmetic default | 8192 | 1152921504606830593, 1152921504606748673, 1152921504606683137 | 128 MiB | +| Guarded review default | 16384 | 1152921504606748673, 1152921504606683137, 1152921504606584833 | 256 MiB | + +Sage 10.9 ran lattice-estimator commit +`53da5982597709ba0fdf94ea37a84d822310fd84`, with `ND.Uniform(-1,1)` and +`ND.CenteredBinomial(21)`. Each profile has 24 estimates: primal uSVP, primal +BDD, dual and dual-hybrid, each with m=N and unbounded independent samples, +under three reduction-cost models. Primal runs use the geometric-series shape +model. The following are minima of reported `log2(rop)` across the selected +attacks and sample counts, **not assigned security levels**. + +| Cost model | Original N=8192 | Review N=16384 | +| --- | ---: | ---: | +| MATZOV classical | 154.40 | 339.59 | +| ADPS16 classical sensitivity | 124.69 | 316.82 | +| ADPS16 quantum-sieving sensitivity | 113.42 | 287.53 | + +The disagreement matters: a historical modulus table or one favorable model +cannot justify calling the original profile 128-bit secure. The larger ring +provides more estimated margin, at a measured cost. It requires fresh keys +and index creation; existing raw defaults remain unchanged for reproducibility. + +Important limits of these estimates: generic LWE models do not certify RLWE +structural security or the circular/key-dependent-message assumptions of the +published s² and automorphism evaluation keys. m=infinity is a sensitivity +case, not a proof that related ring samples are independent. Unlisted attacks +and cost/shape models are not evaluated. The MATZOV source describes a fit +through block size 1024; some larger-ring runs exceed it and extrapolate. +ADPS16 quantum sieving is not a complete quantum attack analysis. Moreover, +Ed25519 receipts are not post-quantum signatures, so the whole guarded protocol +must not be described as post-quantum secure based on an LWE number. +[Lattice-estimator source and interpretation](https://github.com/malb/lattice-estimator), +[current HE parameter guidance](https://homomorphicencryption.org/security-guidelines/). + +Results, all attack outputs, versions and manifest/source hashes are in: + +- `benchmarks/results/native_bfv_parameter_estimates.json` +- `benchmarks/results/native_bfv_review_parameter_estimates.json` + +Reproduce with an independently installed Sage Python and the pinned estimator: + +```bash +git clone https://github.com/malb/lattice-estimator /tmp/xtrace-lattice-estimator +git -C /tmp/xtrace-lattice-estimator checkout 53da5982597709ba0fdf94ea37a84d822310fd84 +.venv/bin/python benchmarks/bfv_parameter_audit.py manifest --profile review \ + --json-out /tmp/bfv-review-manifest.json +DOT_SAGE=/tmp/xtrace-sage /opt/sage/bin/python benchmarks/bfv_parameter_audit.py estimate \ + --manifest /tmp/bfv-review-manifest.json --estimator /tmp/xtrace-lattice-estimator \ + --json-out /tmp/bfv-review-estimates.json +``` + +The Sage executable path is installation-specific. `--profile original` +exports the other profile. No BFV keys are generated or stored by this audit. + +## Conservative correctness bound for this circuit + +`hamming_noise_bound` provides a source-level derivation for independent +review. It uses exact integers and rational numbers with outward rounding, +assuming honestly generated keys, bounded CBD errors, binary owner inputs and +the implemented square-difference/rotate/mask/merge circuit. It is not a +ciphertext validity test, formal proof, or assurance for malicious setup or +arbitrary FHE circuits. The guarded client requires this sufficient bound at +index creation and restoration, separately from receipt validation. + +Let `Q = delta*t + r`, canonical plaintext polynomial coefficients lie in +`[0,t-1]`, and the phase invariant be +`Phi(c) = delta*m + v (mod Q)`, with `||v||_infinity <= V`. +For negacyclic polynomials, use `||a*b|| <= N*||a||*||b||`. + +Fresh encryption has `Venc=(2N+1)*eta`; subtracting two ciphertexts gives +`Vdiff=2*Venc+r`. A gadget switch contributes at most +`S=N*digits*(2^gadget_bits-1)*eta`. For a canonical two-component difference, +write its **integer** phase as `delta*m+v+Q*k`, with + +```text +L = ceil(((N+1)*(Q-1) + delta*(t-1) + Vdiff) / Q) +||k|| <= L +P = N*(t-1)^2 +C = ceil((P+t-1)/t) # Plaintext-product carry bound. +``` + +BFV tensor multiplication computes integer component products before rounding +each by t/Q. Expanding the two integer phases introduces lift terms +`t*(v*k' + v'*k) - r*(m*k' + m'*k)` modulo Q. Omitting them would incorrectly +treat scale-and-round as an operation on phase representatives alone. With +both input bounds equal to Vdiff, a sufficient bound after relinearization is + +```text +Vsquare = ceil( + r*C + delta*r*P/Q + + 2*N*(t-1)*Vdiff + t*N*Vdiff^2/Q + + 2*t*N*Vdiff*L + 2*r*N*(t-1)*L + + (1+N+N^2)/2 +) + S +``` + +The last fraction bounds the three component rounding errors after evaluation +under `1,s,s²`. Let W be the next power of two at least the vector dimension. +Apply `V <- 2V+S+2r` for each of the log2(W) dimension-reduction rotations and +adds. Multiplication by the plaintext selection mask gives +`Vtile=N*(t-1)*V+r*C`. At most K=W tiles enter one response ciphertext, with +K-1 further rotate/add nodes, so +`Vmerged=K*Vtile+(K-1)*(S+2r)`; partial response groups use their actual K. + +Compacting to q' gives +`Vterminal=ceil(q'*Vmerged/Q + (t-1) + (N+1)/2)`. +The extra terms cover the delta-scale mismatch and two component rounding +errors. Exact decryption of every coefficient follows from the sufficient +strict inequalities + +```text +2*(t*Vmerged + (Q mod t)*(t-1)) < Q +2*(t*Vterminal + (q' mod t)*(t-1)) < q' +``` + +For D=512 at the policy's numerical maximum of 65,536 rows, these bounds pass +for both 180-bit-Q profiles. `Vterminal` is 69,633 at N=8192 and 73,729 at +N=16384. This covers more rows than the default index byte cap permits; the +independent resource caps still apply and do not promise every maximum can +fit together. A 120-bit-Q probe passed small random correctness tests but +failed this sufficient full-workload bound, so it was not adopted as the +guarded default. A failing bound means assurance is insufficient; it does +not by itself demonstrate a decryption failure. + +## Performance and validation + +On a Ryzen 7 5800X, Python 3.12.3 and GMP 6.3.0, the final-source benchmark +uses 1,024 vectors of 512 bits, one cold trial and three warm trials. The +following times are warm medians. Setup/import and query encryption are +separate from server evaluation. Within each profile the evaluated ciphertexts +are identical; the two profiles have fresh keys and different ciphertexts. +No network timing or concurrent overlap is claimed. + +| Measurement | Original N=8192 | Review N=16384 | +| --- | ---: | ---: | +| Evaluator search, authentication and serialization | 2.35395 s | 2.43866 s | +| Additional owner-verifier recomputation and receipt | 2.35376 s | 2.43529 s | +| Paired evaluator + verifier time | 4.70963 s | 4.87396 s | +| Python reference distance decoding and parsing | 39.42 ms | 80.29 ms | +| Native distance decoding and parsing | 12.46 ms | 24.65 ms | +| Complete guarded client finish, including receipt and private check | 16.13 ms | 28.43 ms | +| Query encryption and binding | 100.59 ms | 204.76 ms | + +At the larger ring, native decoding is **3.26 times faster** than the Python +reference on the same responses. The evaluator itself is **3.6% slower** than +the original-ring run: doubling N halves the number of packed index tiles in +this workload, offsetting much of the extra arithmetic per tile. These are +local measurements, not a general scaling guarantee. The required second +evaluation dominates the added protocol cost; overall delegated compute is +about doubled, despite preserving the existing fast evaluator. + +One-time key generation takes 4.32/8.87 seconds and index encryption, private +summary construction, export and authentication take 8.08/9.42 seconds for the +original/review profiles. Each evaluator imports setup in roughly 1.73/3.37 +seconds; the verifier has its own comparable import cost. Cold searches take +2.67/3.07 seconds as lazy public key/mask caches are prepared. The benchmark +process, holding both evaluators and the private reference, peaks at +1,094,104/1,985,372 KiB RSS. This is whole-process memory, not a per-server or +locked-secret-buffer measurement. + +| Serialized item, decimal bytes | Original N=8192 | Review N=16384 | +| --- | ---: | ---: | +| BFV public/evaluation keys, per evaluator | 85,602,438 | 171,205,156 | +| Encrypted-index array, per evaluator | 23,598,467 | 23,595,715 | +| Complete authenticated setup, per evaluator | 109,205,199 | 194,805,166 | +| Client query | 368,840 | 737,480 | +| Evaluator response | 102,614 | 205,018 | +| Approval receipt | 198 | 198 | +| Client response + receipt download | 102,812 | 205,216 | +| Client query + response + receipt | 471,652 | 942,696 | +| Additional query + response input to a remote owner verifier | 471,454 | 942,498 | + +The review profile approximately doubles the client exchange and public key +size. Packed index size stays almost constant because fewer larger tiles hold +the same rows; small framing/JSON differences depend on the particular random +keys and ciphertexts. The private summary stays at 16,448 bytes for D=512 and +never goes to either evaluator. Both public evaluators need their own setup. + +Results live in `benchmarks/results/native_bfv_guarded_original_1024.json` and +`benchmarks/results/native_bfv_guarded_review_1024.json`. JSON contains synthetic +workload dimensions, timings, sizes and source/binary hashes, with no keys, +ciphertexts or private summaries. + +```bash +.venv/bin/python benchmarks/bfv_guarded.py --profile original \ + --json-out /tmp/bfv-guarded-original.json +.venv/bin/python benchmarks/bfv_guarded.py --profile review \ + --json-out /tmp/bfv-guarded-review.json +.venv/bin/python -m pytest tests/x_vec/test_bfv_guarded_client.py \ + tests/x_vec/test_bfv_private.py tests/x_vec/test_bfv_assurance.py -q +``` + +For a remote owner verifier, each search additionally sends it the query and +proposed response, on top of provisioning a second copy of public keys/index. +The client receives the 198-byte receipt. Counting only those 198 bytes as the +cost of the entire distributed protocol would omit that additional traffic +and full recomputation. If verification runs locally, the owner instead needs +the public setup and compute resources locally. The larger ring separately +changes query, response and key sizes. + +Validation of the final implementation: + +- **351 offline tests pass:** 323 SDK tests, including 269 BFV cases, plus + 28 retained SEAL experiment tests. Production HTTP integrations and GPU + hardware tests are excluded. All 49 new security/arithmetic/bound cases pass. +- **183 binding/security tests pass under AddressSanitizer and + UndefinedBehaviorSanitizer.** One OS memory-lock test is deliberately skipped + because ASan intercepts `mlock`; it passes with the release binary and a + zero-lock-limit child process. Both sanitized extension paths are explicitly + loaded, preventing an accidental release-binary fallback. LeakSanitizer is + disabled in this environment because execution uses ptrace. +- The standalone private arithmetic/wipe test passes ASan/UBSan. The GCC 13.3 + and Clang 18 optimized secret-taint runs each report zero errors and no heap + leaks; both deliberately branching negative controls fail as required. +- SDK and BFV benchmark lint/type checks pass. The platform wheel contains + both native modules and the new Python APIs. Documentation builds offline + with the pre-existing SEAL research-link warning. The documented N=16384 + example, including guarded state restoration, passes from the extracted wheel. + +The benchmark JSON records base revision `0cd7b41` with a dirty working tree; +the recorded source/binary hashes identify the measured implementation and +match the final files. Estimator outputs likewise match their pinned manifests +and runner hash. These checks are reproducible engineering evidence, not an +independent security audit. + +## Remaining production gates + +| Area | Evidence added here | Remaining requirement | +| --- | --- | --- | +| Malicious-result oracle | Exact public recomputation receipt checked before private work; old attack blocked in guarded regression | Independent protocol review and actual owner-verifier isolation, pinning and key custody; no general CCA2 or succinct proof claim | +| Private implementation | Separate fixed-work decoder, locked/wiped native buffers, GCC/Clang taint tests, arithmetic oracles and sanitizers | Full private key generation/encryption/import/summary path, compiler/CPU review, Python copies and deployment side channels | +| Parameters and failures | Pinned actual-distribution estimates and conservative circuit-specific bounds; larger review profile | Independent analysis of models, omitted attacks, ring/evaluation-key assumptions and bound derivation | +| Service integrity/availability | Existing per-object tickets, caps and admission; guarded exports bind verifier pin | Durable anti-rollback/replay and quotas, owner authorization at verifier, tenant/process isolation and transport limits | +| Application leakage | Explicitly unchanged all-distance interface | Review chosen-query capability, result-dependent fetches, IDs/sizes/timing; private top-k/PIR and circuit privacy remain unimplemented | + +The implemented trust boundary materially changes which ciphertexts may reach +decryption, while these deployment and cryptographic assurance tasks still +prevent an honest production-security sign-off. diff --git a/docs/research/native-bfv-residue.md b/docs/research/native-bfv-residue.md new file mode 100644 index 0000000..81d904b --- /dev/null +++ b/docs/research/native-bfv-residue.md @@ -0,0 +1,277 @@ +# Persistent RNS BFV server + +The measurements below describe the initial residue backend. The follow-up +[fused NTT and RNS scaling report](native-bfv-fused-rns.md) documents the current +default kernels and ABI 4; the earlier kernels remain available as controls. + +`BFVClient(rns_modulus=True, server_backend="residue")` keeps ciphertexts as +arrays of 60-bit residues after the initial BFV tensor scale-and-round. Key +switching, rotations, additions, plaintext masks and the result merge tree run +on these arrays in our own C++. The reference, optimized, `rns` and `native` +backends can evaluate the same keys and ciphertexts and give exactly the same +output coefficients. The existing Paillier and SEAL implementations are retained. + +## Configuration and compatibility + +This is an explicit new cryptographic configuration. It requires **fresh keys +and a fresh encrypted index**. Existing prime-modulus keys cannot be used with +`residue`; they continue to work with the previous backends. Nothing migrates +keys or ciphertexts automatically. + +The default remains `rns_modulus=False` and `server_backend="optimized"`. +When the RNS option is off, parameter serialization and key fingerprints retain +their original representation, including compatibility with saved v1 keys. +When enabled, `rns_modulus: true` is serialized in the configuration and public +key parameters and included in the fingerprint. Ciphertexts keep the same +packed format and contain the actual Q and key identity. + +The first residue layout supports ciphertext modulus sizes 60, 120, ..., 480 +bits. For each ring degree, it selects distinct primes below 2^60 with +`p = 1 mod 2N`, in the same deterministic order as the auxiliary NTT bases. +The default 180-bit Q is the product of three such primes. The binary gadget, +error sampling, plaintext modulus, batching, encryption and exact rounding +algorithms are unchanged. An equal-bit response-compaction request retains Q +instead of increasing it to the same-bit prime. + +This remains experimental leveled BFV with no independent audit or security-level +claim; see the [internal security review](native-bfv-security.md). It uses no SEAL +code and performs no bootstrapping. The performance +comparison below does not establish equivalent security to SEAL's parameters. + +## Build and use + +The extension interface is now ABI 3. Rebuild an older binary: + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +``` + +```python +import json +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient + +client = BFVClient(embed_len=512, rns_modulus=True, server_backend="residue") +server = BFVClient(skip_key_gen=True, server_backend="residue") +server.load_config(json.loads(client.stringify_config())) +server.load_stringified_keys(client.stringify_pk()) +assert server.keys is None + +query = [0, 1] * 256 +vectors = [query, [1 - bit for bit in query]] +encrypted_query = client.encrypt_vec_one(query) +encrypted_index = client.encrypt_vec_packed(vectors) +response = server.encode_hamming_server_packed(encrypted_query, encrypted_index, 2) +assert client.decode_hamming_client_packed(response, 2) == [0, 512] +``` + +Both individual and packed search methods use the new server. Reuse a server +across queries to amortize public-key and mask preparation. Key generation, +encryption and client decryption continue to use our Python/GMP scheme. +Top-k selection still happens on the client. These local interfaces do not add +BFV support to the production HTTP endpoints. + +## Why the representation matters + +Previously, auxiliary primes only accelerated polynomial products. Their +product M differed from the ciphertext modulus q. Every key switch had to +convert both output components back to GMP integers and reduce them modulo q. +Rotations, additions and the merge tree then operated on those GMP objects. + +With Q equal to the product of the ciphertext primes, the Chinese Remainder +Theorem identifies arithmetic modulo Q with componentwise arithmetic modulo +each prime. A key-switch product needs only these primes, regardless of the +unreduced integer convolution's size: its result is needed modulo Q. The +default key-switch basis shrinks from four primes to three. Ciphertext additions +and automorphisms become small-word loops; masks multiply directly in this +basis. This follows the residue representation described in the textbook's +[RNS operations section](https://fhetextbook.github.io/ApplyingRNSTechniquestoFHEOperations.html). + +Binary gadget decomposition still needs the unique coefficient in [0,Q). +`ResidueArithmetic::compose_words` reconstructs it with exact mixed-radix +(Garner) arithmetic, using fixed 64-bit limbs and 128-bit products. It neither +allocates GMP integers nor estimates a CRT quotient with floating point. For +residues r_j, it finds mixed-radix digits d_j by repeatedly computing +`(value - d_k) * inverse(p_k) mod p_j`, then forms +`d_0 + p_0*(d_1 + p_1*(...))`. Each d_j is in [0,p_j), so this is the canonical +representative. Its binary gadget digits are the same as the reference's. +Digits up to 60 bits are extracted once and reused across primes. Wider gadgets +are reduced in chunks of at most 60 bits; the maximum supported Q uses eight +64-bit limbs. No additional evaluation keys or special key-switch prime are +introduced. + +The scale-and-round in ciphertext multiplication still requires the signed +integer tensor before reducing modulo Q. It uses the existing sufficiently +large auxiliary base, exact GMP CRT and signed rounding. After that step, the +ciphertext remains in RNS until the final response is composed and compacted. +This preserves exact coefficient equality with our reference, including its +rounding choices and noise behavior for the same key. Ciphertexts normally use +coefficient-form residues between operations; transforms are applied for +polynomial multiplication, rather than keeping every operation in NTT form. + +## Reproduce the comparison + +```bash +.venv/bin/python benchmarks/bfv_server.py --num-vectors 1024 \ + --rns-modulus --backends native,residue --seal --repeats 4 \ + --native-profile --json-out /tmp/bfv-residue-1024.json +``` + +Use `--num-vectors 32` for the small workload. Omit `--seal` when TenSEAL is +unavailable. `benchmarks/bfv_client_matrix.py` also accepts `--rns-modulus` and +`--server-backend residue`. Native backends receive identical public keys and +encrypted inputs; every result must match exactly and decrypt to all expected +Hamming distances. SEAL gets the same plaintext vectors with its own parameters. +The first search includes lazy cache setup; three warm searches follow, with +backend order alternating. Additional profiled searches are excluded from +timing medians. Run without concurrent tests or benchmarks. + +## Recorded results + +The final comparisons and phase profiles are recorded below. All times are +seconds on the AMD Ryzen 7 5800X, Python 3.12.3, GMP 6.3.0, GCC 13.3.0 (`-O3`) +and TenSEAL 0.3.16. There is one CPU evaluation thread, with no GPU or SEAL code +in our backend. Each native row uses N=8192, t=65537, a 180-bit product Q, six +30-bit gadget digits, error eta=21 and 50-bit terminal compaction. + +| Vectors | Backend | First search | Median of three warm searches | +| ---: | --- | ---: | ---: | +| 32 | `native`, same product Q | 0.5182 | 0.1591 | +| 32 | `residue` | 0.3970 | 0.0873 | +| 32 | SEAL prototype | 0.0772 | 0.0775 | +| 1,024 | `native`, same product Q | 5.6354 | 5.2780 | +| 1,024 | `residue` | 3.1145 | 2.8064 | +| 1,024 | SEAL prototype | 2.4680 | 2.4814 | + +At 1,024 vectors, residue takes **46.8% less server time** than native with the +same keys and inputs: a **1.88x speedup**. Its warm time is **13.1% longer** than +SEAL's in this run. The small workload improves by 45.1%. Every ciphertext from +our two backends matches exactly, including the compacted response. Every +decoded distance matches the plaintext oracle for every backend. + +SEAL uses active coefficient-prime sizes [43,43,44,44] and an additional 44-bit +special key-switch prime, with its TC128 setting. Our scheme has different +parameters and key switching, with no matching security-level claim. Native +server times include framing, ciphertext decoding/encoding, arithmetic and +compaction, but exclude outer MessagePack. The preserved SEAL prototype also +includes its MessagePack and temporary-file binding I/O. Key import, encryption, +client decryption and network time are excluded. These are practical server +calls, rather than timings of identical cryptographic kernels. + +The 1,024-vector response is 102,496 bytes and query plus response is 471,250 +bytes, matching the preceding native experiment's measured sizes. The encrypted +index is 23,598,495 bytes and serialized public keys are 85,603,326 bytes. This +change targets compute, with the same packed wire structure. Small byte-count +variation between fresh keys/runs is possible. Both native backends have a +26-bit minimum response noise budget here; SEAL reports 19 bits for its own +parameters and noise diagnostic. + +Prepared C++ key payload falls from 56,623,104 to 42,467,328 bytes (25% less), +with the same 18 keys. This is an additional runtime cache, not the serialized +BFV setup-key download. Transform-table payload is 2,752,512 bytes in either +backend; prepared plaintext tables/masks use 655,360 versus 589,824 bytes. +These figures exclude containers, small CRT constants and transient allocations. + +Raw samples, environment, exact modulus/primes, ciphertext checks and source/ +binary hashes are in `benchmarks/results/native_bfv_residue_32.json` and +`benchmarks/results/native_bfv_residue_1024.json`. + +An additional 1,024-vector control kept the original prime-modulus configuration +and `native` backend: first search 5.8559 seconds, warm median 5.4560 seconds. +Its raw data is in `benchmarks/results/native_bfv_residue_prime_control_1024.json`. +This is a separate run with fresh random encryption, not part of the interleaved +same-key comparison used to calculate the 1.88x speedup. The earlier recorded +5.3426-second native result is retained in [the previous report](native-bfv-server.md). + +## What remains slower than SEAL + +The additional exclusive phase profile for 1,024 vectors shows the effect of +retaining residues. These are instrumented extra searches, excluded from the +performance medians above. + +| Phase | `native` | `residue` | +| --- | ---: | ---: | +| Forward NTT | 1.5804 | 1.2114 | +| Inverse NTT | 0.5974 | 0.4784 | +| GMP CRT reconstruction/conversion | 0.7341 | 0.1790 | +| Separately instrumented reduction modulo Q | 0.4763 | 0 | +| Fixed-word RNS composition | 0 | 0.0589 | +| Gadget extraction/conversion | 0.0637 | 0.0816 | +| Automorphisms | 0.3657 | 0.0311 | +| Additions/subtractions | 0.1871 | 0.0586 | +| Pointwise products/accumulation | 0.4527 | 0.2873 | +| Scale-and-round | 0.1639 | 0.1625 | + +Both perform 703 key switches. Forward transforms fall from 18,280 to 13,934; +inverse transforms fall from 7,480 to 5,946. The residue path has 194 GMP CRT +operations: three tensor components for each of 64 tiles, plus two final +response components. The earlier native path has 1,726. Reductions inside the +remaining tensor/compaction work are charged to their own scopes, so the zero +`mod_q` entry does not mean that all big-integer modular arithmetic is gone. +The full JSON retains the other phases and exact call counts. + +Forward/inverse NTT now account for about **60%** of our profiled time. The +remaining GMP CRT and scale-and-round account for about **12%**. The fixed-word +reconstruction needed for binary gadget extraction is about **2%**. Python, +binding and unscoped work together account for about **3%**. + +Source inspection points to two concrete areas for the next optimization: + +- SEAL's [NTT implementation](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/ntt.cpp) + and [transform handler](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/dwthandler.h) + use transform-ordered root tables and specialized/unrolled Harvey butterflies. + Our kernels still use a separate negacyclic twist pass and generic cyclic + butterfly loops with strided root access. Tuning that kernel now targets most + of our search time. The installed SEAL wheel's optional acceleration features + have not been established; these results do not assume Intel HEXL is enabled. +- SEAL's [BFV evaluator](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/evaluator.cpp) + and [RNS tools](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/rns.cpp) + perform BEHZ-style multiplication with RNS base conversion, division and + scaling. Our tensor scale-and-round still reconstructs exact GMP integers. + Replacing that boundary requires preserving BFV's rounding/noise semantics; + dividing each residue independently would be incorrect. + +These are source-based explanations and measured bottlenecks in our code. +We have not instrumented SEAL's internal phases, so we cannot assign an exact +fraction of the remaining 0.325-second gap to each difference. Changes to +transform kernels and RNS multiplication can be measured independently using +the preserved native engine as an oracle. Further changes to private key +generation or encryption would not affect this server-search metric. + +## Implementation files and verification + +| Path relative to the repository | Responsibility | +| --- | --- | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_residue.h` | Persistent RNS ciphertexts, fixed-word reconstruction and complete server | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/rns_ntt.h` | Shared NTT/key dot products and optional ciphertext-prime basis | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bfv_server.h` | Shared layout, masks and streaming merge order | +| `src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext/bindings.cpp` | Validated ABI 3 boundary and native plan selection | +| `src/xtrace_sdk/x_vec/crypto/encryption/bfv.py` | Optional product modulus and compatible key serialization | +| `src/xtrace_sdk/x_vec/crypto/encryption/bfv_rns.py` | Compiled arithmetic context and key preparation | +| `src/xtrace_sdk/x_vec/crypto/bfv_client.py` | Client configuration and backend selection | +| `tests/x_vec/test_bfv_residue.py` | Saved-key compatibility, parameters, exact search comparisons | +| `tests/x_vec/native/test_bfv_rns.cpp` | Independent integer/gadget oracles across all supported Q widths | + +The common native boundary tests also run with the residue backend: malformed +inputs, public-only evaluation, direct packed buffers, concurrent searches, +profiling, cache invalidation and capsule lifetimes. Standalone tests compare +mixed-radix reconstruction, gadget digits, key switching and rotations with +GMP and schoolbook integer oracles, including zero, Q-1, prime/word boundaries, +one-bit gadgets, wide gadgets and 480-bit Q. They also assert that a residue +key switch performs no GMP CRT reconstruction or reduction modulo Q. + +Validation passed 192 offline tests, including 110 BFV tests; production API and +GPU tests were excluded. Ruff lint, mypy, and formatting checks for the BFV +modules, tests and benchmarks passed. A broader SDK formatting check reports +existing formatting differences in other modules and unchanged portions of +`xtrace_types.py`; those unrelated sections were left alone. Standalone +arithmetic and Python-binding AddressSanitizer/UndefinedBehaviorSanitizer runs +passed; the latter ran 25 server, compatibility and boundary tests against the +sanitized binary. The local tracing sandbox requires leak checking to be +disabled, as in the previous native-backend verification. + +A built CPython 3.12 platform wheel includes the new header and ABI 3 extension. +An isolated import from that wheel passed public-only residue search and exact +native fallback comparison, without importing SEAL or TenSEAL. +The default N=8192 example above and both matrix-benchmark interfaces passed. +Sphinx built successfully with the existing README cross-reference warning in +`bfv-packed-hamming.md`; no new documentation warning was introduced. diff --git a/docs/research/native-bfv-security.md b/docs/research/native-bfv-security.md new file mode 100644 index 0000000..3a1f700 --- /dev/null +++ b/docs/research/native-bfv-security.md @@ -0,0 +1,362 @@ +# Internal BFV security review — 2026-09-10 + +**Decision: keep this implementation experimental. It is not ready for a +production security claim.** The arithmetic and input handling have stronger +test coverage, but protocol integrity, parameter assurance and private-key +implementation assurance remain open. This is an internal, AI-assisted source +review and regression exercise, not an independent cryptographic audit, +penetration test of a deployed service, or proof of security. + +**2026-09-11 follow-up:** the optional +[authenticated, result-checked session layer](native-bfv-verified-sessions.md) +adds whole-setup authentication, one-use requests, a private Hamming result +check, encrypted private exports and bounded admission. It also demonstrates +that observable acceptance/rejection can still recover the secret key even +with those checks. The findings below describe the original raw BFV APIs; +the follow-up records the new mitigations, their performance and the remaining +production blockers. + +**Later 2026-09-11 follow-up:** [approval before decryption](native-bfv-predecryption.md) +adds a guarded API with owner-controlled public recomputation, a separate +private C++ decoder, pinned lattice estimates and conservative circuit bounds. +It conditionally blocks the demonstrated oracle at an additional trust and +compute cost. The historical findings below remain applicable to the raw APIs. + +The optimization baseline reviewed is commit `06d8d17`, following +`e5c0751`. This report and the accompanying hardening are on +`research/bfv-packed-hamming`. No production endpoint or `main` change is part +of this work. The Paillier implementations and SEAL experiment are preserved. + +## Scope and trust model + +The review covers `crypto/encryption/bfv.py`, `bfv_evaluator.py`, `bfv_rns.py`, +`bfv_native.py`, `crypto/bfv_client.py`, the BFV types in +`x_vec/utils/xtrace_types.py`, and every source header plus `bindings.cpp` in +`crypto/bfv_cpu_ext/`. It follows key generation, encryption, public evaluation, +packed serialization, response compaction, decryption, input checks and native +ownership/concurrency. Existing BFV tests, build rules and the benchmark's +secret/public separation were also inspected. Paillier, SEAL internals, +production HTTP services, dependency implementations and deployment +infrastructure are outside this review's audit scope. + +The assumed data owner controls a trusted private client and its secret key. +The evaluator receives public encryption/evaluation keys, encrypted index and +encrypted queries. An adversary may replace, replay, reorder or corrupt the +server's output or network messages. A separate honest-but-curious model assumes +the server follows the specified circuit and only observes its inputs and +execution. Local side-channel attackers are considered a deployment concern, +but no timing, cache, power or fault-injection exploit was attempted. + +The local BFV API supplies no authenticated network session, tenant policy, +query identifier, index-version binding or verified computation. The wire's +key ID is public. Decrypted distances and top-k selection are private-client +operations. A later document fetch, telemetry event, error response or retry +can cross that privacy boundary even if this arithmetic module never makes a +network call. + +## Findings + +Severity below describes impact **if the stated integration exposes the +relevant surface**. These are not claims that a deployed XTrace endpoint is +currently exploitable. + +| ID | Severity / status | Finding and precondition | +| --- | --- | --- | +| BFV-01 | High; open deployment blocker | An attacker able to observe chosen-ciphertext decryptions can recover the secret. Error/noise feedback must also be treated as secret-dependent. | +| BFV-02 | High; open for an untrusted evaluator | Ciphertext validation and distance range checks do not establish correct computation or prevent replay. | +| BFV-03 | High; open assurance blocker | Algebraically accepted parameters have no established security level or negligible failure-probability guarantee. | +| BFV-04 | High with a local observation surface; open | Private Python/GMP operations are variable time, with no secret-memory erasure guarantee. Secret serialization is plaintext. | +| BFV-05 | Medium; locally hardened | Oversized byte/hex fields were converted before size rejection; key JSON lacked an import-size bound and admitted ambiguous encodings. Service-wide resource limits remain necessary. | +| BFV-06 | High if setup is unauthenticated; documented, open | The key fingerprint does not authenticate the public bundle and does not cover evaluation keys. | +| BFV-07 | Medium; open protocol policy | Distances, query-derived access, counts and stable identifiers have application-level leakage; shared-key query access does not enforce database confidentiality from that query holder. | + +### BFV-01: a decryption oracle can disclose the key + +For our formula `Decrypt(c) = round(t*(c0+c1*s)/Q) mod t`, construct +`c0=0` and the constant polynomial `c1=floor(Q/t)`. This requires only public +parameters and the public key ID. For a ternary secret and the test parameters, +decrypting yields each secret coefficient modulo t; mapping `t-1` to `-1` +recovers the entire secret. The regression test demonstrates this with a fresh +local key and no attack on an external system. The small ring makes the test +fast; the construction is algebraic, not a brute-force attack on its dimension. + +`test_decryption_oracle_would_reveal_secret_key` intentionally **passes when +this disclosure is possible**. It records a forbidden integration pattern; +it is not a mitigation test. The demonstration uses the raw polynomial +decryption API, not a claim that one call to the range-checked Hamming decoder +returns the full key. Rejecting some decoded values can itself create a +secret-dependent success/failure oracle if that outcome is observable. + +Do not expose `BFV.decrypt`, decoded distances, `noise_budget`, per-response +errors or decryption-dependent telemetry to the untrusted evaluator. A +production protocol must analyze any result-dependent follow-up, including +which document IDs are fetched. SEAL's own guidance likewise treats decryptor +outputs and noise diagnostics as private, and Apple's BFV guidance explicitly +warns about decryption-oracle key recovery. +[SEAL security guidance](https://github.com/microsoft/SEAL/blob/main/SECURITY.md), +[Apple BFV guidance](https://github.com/apple/swift-homomorphic-encryption/blob/main/README.md#homomorphic-encryption-he). + +Client response parsing now limits accepted Hamming responses to two +components at the configured terminal modulus or original Q, before private +decryption. This rejects unexpected protocol shapes. **It does not provide +chosen-ciphertext security.** Blanket rejection of all-zero ciphertexts would +also reject legitimate homomorphic results and would not prevent an attacker +from constructing other chosen ciphertexts. + +### BFV-02: plausible results and replays remain unauthenticated + +Two tests show that the client accepts both a transparent zero ciphertext and +a fresh public-key encryption of the plausible distance 1. Neither proves +that the encrypted query was evaluated against the intended index. A third +test replays the response to one query while the correct answer to another +query is different. Key identity, shape and `0 <= distance <= embed_len` +checks all still pass. There is no query or index epoch in the response frame. + +Authenticated transport and authenticated setup are necessary to exclude an +outside message replacement attack. Binding messages to a tenant, key, query +nonce, index version, ordered IDs, layout and parameters can exclude accidental +mixups and outsider replay when those bindings are authenticated. **A signature +from the evaluator alone does not prove it computed the right circuit.** An +actively malicious evaluator requires a reviewed verifiable-computation or +other protocol mechanism, or an explicit weaker trust assumption. No bespoke +authentication scheme was added around homomorphic ciphertexts in this change. +SEAL also distinguishes loading/decrypting a ciphertext from authenticating it. +[SEAL ciphertext authenticity guidance](https://github.com/microsoft/SEAL/blob/main/SECURITY.md#ciphertext-authenticity). + +### BFV-03: parameter and failure assurance is incomplete + +`BFV.validate_parameters` checks algebra, supported bounds and types. It +deliberately accepts insecure tiny rings for tests, and also accepts large +Q/small-N combinations that must not be selected for production merely because +the method returns successfully. The API does not advertise a 128-bit security +setting, enforce a deployment allowlist or estimate lattice attacks. + +The optimized benchmark configuration is: + +| Item | Actual construction | +| --- | --- | +| Ring | `Z_Q[X]/(X^8192+1)` | +| Q | Product of three 60-bit NTT primes; 180 total bits | +| Plaintext | t=65537, two-row batching | +| Secret and encryption ephemeral u | Independent uniform coefficients in `{-1,0,1}` | +| Error | Centered binomial, eta=21; variance 10.5 and standard deviation about 3.240 | +| Evaluation keys | Six 30-bit gadget digits for s² and required automorphic secrets, under the same secret | +| Terminal response | Exact componentwise rounding to a 50-bit prime | + +Q's three primes are 1152921504606830593, 1152921504606748673 and +1152921504606683137. They are the actual encryption modulus; the larger tensor +NTT basis is only an arithmetic workspace. Counting its product as the RLWE +encryption modulus would be incorrect. + +The older HE standard's classical ternary-secret table lists log2(Q)=218 at +N=8192 for its 128-bit target and specified error assumptions. Our Q=180 is +below that reference bound, but our binomial sampler and secret-dependent +evaluation keys still require a justified analysis. This table comparison +**does not establish security for our implementation**. +[HE Standard v1.1, Table 1, ternary section on PDF page 27](https://homomorphicencryption.org/wp-content/uploads/2018/11/HomomorphicEncryptionStandardv1.1.pdf). +The consortium points to newer parameter guidance; parameter selection must +cover both attacks and functional correctness. +[Current guidance index](https://homomorphicencryption.org/security-guidelines/), +[Security Guidelines for Implementing Homomorphic Encryption](https://cic.iacr.org/p/1/4/26). + +Before making a security-level claim, record an expert-reviewed configuration +with concrete classical/quantum attack estimates, estimator version and cost +models, the exact secret/error distributions, sample exposure and the +assumptions needed by the evaluation keys. A modern estimator was not run in +this review. Its rough and full estimators can use different cost models; +one convenient output number is insufficient evidence. +[Lattice estimator documentation](https://github.com/malb/lattice-estimator). + +Correctness also needs a circuit-specific bound on decryption failure, +including compaction, maximum dimensions/merge depth, message patterns, +key distributions and repeated queries. The tests cover fresh randomness, +extreme patterns and exact arithmetic boundaries. They cannot establish a +negligible failure probability. The benchmark's 26-bit noise budget measures +remaining noise margin on those results; it is neither 26-bit nor 128-bit +cryptographic security, and positive noise diagnostics cannot rule out an +earlier wraparound. + +### BFV-04: private operations and key storage + +Key generation, ephemeral/error sampling, encryption and decryption use +`secrets` and Python/GMP. The review found no benchmark PRNG feeding production +encryption: public uniform coefficients use `secrets.randbelow(Q)`, ternary +coefficients use `randbelow(3)`, and each binomial sample subtracts the popcounts +of two independent `randbits(eta)` results. Tests check mapping and bounds and +verify that an entropy-source exception aborts key generation/encryption. +They do not constitute an entropy audit or statistical security proof. + +Python objects, GMP multiplication/packing, comparisons, allocation and error +paths have no constant-time contract. We did not measure an exploitable timing +channel, but cannot claim its absence. Moving **public server** evaluation to +constant-width RNS does not harden the private key operations. The server +kernels receive no secret key; tests enforce that public-only evaluation never +calls private phase computation. + +`stringify_sk` is an explicit plaintext export. Secret tuples, GMP limbs and +JSON copies have no guaranteed erasure, locked-memory protection or encrypted +storage. Key custody, crash dumps, logs, process isolation and backup handling +need a deployment design. Do not claim that a Python zeroization helper would +erase every copy. Private arithmetic requires independent implementation and +side-channel review appropriate to the intended platform. + +### BFV-05: input handling fixes and remaining resource policy + +The review found that the original wire readers called `int.from_bytes` on +every field before checking size. Arbitrarily long high-order zero padding +could therefore consume conversion memory while representing a small valid +number. Public-key coefficients and native modulus text were similarly passed +to GMP before their bit limits were established. Key JSON had no pre-parse +size cap, silently accepted duplicate object fields, and accepted boolean +version values and multiple textual encodings of some integers. + +The hardening adds: + +- A shared Python wire reader with per-field byte and integer limits checked + before big-integer conversion/unpacking. Negative values, bools and floats + are rejected. C++ still checks every imported coefficient is strictly below + Q before any arithmetic relies on the convolution bound. +- Public-key and secret-key JSON limits of 128 Mi and 1 Mi **characters**, + respectively, checked before `json.loads`. These are local resource policy, + not security parameters. `BFV.deserialize_*` accepts an explicit `max_chars`; + `BFVClient.load_stringified_keys` exposes `max_public_key_chars` for larger + trusted experiments. Existing default public bundles (~86 MB ASCII) fit. +- Duplicate-field rejection, bounded canonical lowercase coefficient hex, + canonical decimal Galois exponents, strict key fields/version types, and + normalization of excessive JSON nesting into `ValueError`. +- Native modulus-text bounds before GMP parsing and a modulus-type check on + direct Python ciphertext objects. Failed key imports leave the client and + existing evaluator context unchanged. + +Previously emitted v1 keys and minimally encoded ciphertexts remain readable; +the serialization version and cryptographic values are unchanged. Noncanonical +hand-written JSON, excessive zero padding and over-limit bundles are now +rejected intentionally. JSON whitespace within the overall cap remains valid. + +These caps do not bound a service's total memory or CPU use. JSON parsing still +allocates the entire permitted object tree. A character cap is not a byte cap +for Unicode; a service must restrict transport bytes before buffering or +decoding, authenticate and allowlist setup parameters, limit index/request +sizes and concurrent evaluations, and impose process/resource quotas. Direct +Python object APIs and private C++ capsules assume trusted application code; +they are not sandbox boundaries. GMP out-of-memory behavior was not fault +injected, and the native extension's exception handling is not an OOM survival +guarantee. + +### BFV-06: authenticate the whole setup bundle + +`_key_id` hashes serialized parameters plus encryption polynomials `(b,a)`. +It is useful for key mixup detection. It is unkeyed, and omits relinearization +and Galois keys. A test changes a relinearization coefficient, successfully +loads the bundle with the same key ID, and confirms the whole-bundle digest +changed. An attacker able to replace all of `(b,a)` can also recompute a new +fingerprint; neither equality check establishes ownership. + +Public import cannot verify the secret-dependent algebraic relationship of +evaluation keys from their shape alone. Secret import checks ternary +coefficients and consistency with the public RLWE sample, but does not certify +key generation or authenticate the public setup. Provision and authenticate +the complete bundle, configuration and intended owner from a trusted source. +An unkeyed digest is meaningful only when its expected value is authenticated +out of band. No incompatible fingerprint change was made in this review. + +### BFV-07: application leakage and authorization + +This protocol returns all candidate distances in packed ciphertexts and lets +the client compute top-k. It does not reveal only three distances. A party with +chosen-query access to all Hamming distances can recover a binary database +vector v: querying the zero vector gives its weight w; querying unit vector +e_i gives `w+1-2*v_i`. These D+1 queries determine every bit. For a data owner +searching their own data this is expected; it is not an access-control scheme +for mutually distrustful users sharing the key or query capability. + +The server also observes vector count, parameters, encrypted-index identity, +message sizes and query timing. Subsequent top-k document fetches may reveal +access patterns. Ciphertext randomization does not hide these metadata. No +circuit-privacy, multi-tenant isolation, forward secrecy or private retrieval +guarantee is established here. Define which leakage is acceptable and enforce +owner/key authorization in the application. + +## Evidence and limits of testing + +The new `tests/x_vec/test_bfv_security.py` contains **63 cases**. Coverage +includes the four explicit protocol-limit demonstrations, randomness handling, +public-only operation, parser limits/ambiguity, response shape before private +decryption, atomic key-load failure, 450 deterministic wire mutations across +kernel levels, and 100 native coefficient-buffer cases. Mutation tests compare +accepted attacker-modified ciphertexts with the GMP reference and reject +malformed framing; they do not assume ciphertext tampering must fail. + +A default-size stress test uses two fresh N=8192, Q=180 RNS key sets, six query +patterns (zeros, ones, alternating bits, first/last one-hot and random bits), +and a 33-vector index with a partial tile and merge. Every distance is checked +against plaintext Hamming distance. Fixed PRNGs generate only test vectors and +mutation positions, never cryptographic randomness. + +The standalone C++ suite checks old/fused NTT spectra through N=32768, +schoolbook products, certified CRT boundaries, gadget widths through the +supported maximum, and exact signed RNS scaling at all eight Q widths. The +review extended the scaling oracle to plaintext multipliers immediately below +2^60 where Q permits. Fixed-word corrections are exercised at ambiguous +rounding boundaries, not merely random points. + +The source review checked the 60-bit-prime, 64/128-bit accumulator and +eight-/18-word storage bounds, the signed tensor quotient bound, immutable +server/key ownership while the GIL is released, byte-buffer lifetime, canonical +imports and guarded exports. No memory corruption was found in the exercised +paths. This is not formal verification, an exhaustive fuzzer campaign or an +independent proof of the RNS construction. + +Validation on this branch: + +- **263 offline tests pass**, including **181 BFV tests** and the existing + Paillier/SEAL coverage. HTTP integration and unavailable GPU tests were excluded. +- The C++ arithmetic suite passes AddressSanitizer and UndefinedBehaviorSanitizer. +- **96 Python-binding/security tests pass** against an ASan/UBSan extension. + This includes malformed inputs and the real-default stress case. +- Ruff and mypy pass for the SDK and BFV benchmark code. CI runs the new + security tests and includes them in the sanitized binding run. + +Reproduce the principal security checks: + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +.venv/bin/python -m pytest tests/x_vec/test_bfv_security.py -q +``` + +The exact standalone and binding sanitizer commands are in +`.github/workflows/ci.yml`. Sanitizer success covers the executed cases on this +compiler/platform; it does not establish constant-time behavior or memory +erasure. Test output contains no generated secret key material. + +The final performance check after hardening used the same-key interleaved +1,024-vector × 512-dimension workload as the optimization report. Warm medians +were 2.7874 s for the previous residue kernels, 2.4236 s for the new kernels +and 2.4850 s for the preserved SEAL prototype. Native ciphertexts were identical +and all distances were correct; response size stayed 102,496 bytes. This +retains a 13.1% reduction in server time. Raw samples and final source/binary +hashes are in `benchmarks/results/native_bfv_security_1024.json`. The +[performance report](native-bfv-fused-rns.md) explains the different SEAL +parameters, timing boundary and absence of an equal-security comparison. + +## Production gates + +1. Have an independent HE cryptographer review the construction, exact RNS + scaling proof, distributions, parameter estimates, evaluation-key assumptions + and a complete circuit-specific decryption-failure analysis. Freeze and + enforce a reviewed configuration separately from algebra-test parameters. +2. Specify the end-to-end threat model and every output channel. For an active + server, provide a reviewed integrity/verifiability design and demonstrate + that response handling, errors, retries and document retrieval do not expose + a harmful decryption oracle. Authenticate setup and bind sessions, queries, + index versions, IDs and layouts. +3. Review private arithmetic and key custody for the deployment platform. + Establish appropriate side-channel controls, logging/storage policy, + isolation, rotation and incident handling. +4. Apply service-level resource limits and authorization. Run sustained + adversarial fuzzing, platform/compiler coverage, dependency review and load + tests on the actual endpoint before rollout. + +Passing the current tests closes specific regression risks. It does not close +these gates or make the homemade BFV interchangeable with an independently +reviewed production cryptosystem. diff --git a/docs/research/native-bfv-server.md b/docs/research/native-bfv-server.md new file mode 100644 index 0000000..9b85cee --- /dev/null +++ b/docs/research/native-bfv-server.md @@ -0,0 +1,253 @@ +# Complete C++ BFV server evaluation + +`BFVClient(server_backend="native")` evaluates a complete encrypted Hamming +search inside our own C++ implementation. Python passes packed ciphertext +buffers in and receives packed result buffers out. Masks, polynomial operations, +rotations, the result merge tree, and terminal modulus compaction run in C++. +Key generation, encryption and decryption continue to use `encryption/bfv.py`. + +This is an optional server backend with the same parameters, keys, packing and +wire format as the earlier native BFV implementation. The `optimized`, +`reference` and `rns` backends remain available. It uses GMP and the C++ standard +library, and does not compile or link SEAL. Like the existing scheme, this is +experimental leveled BFV without a security audit or bootstrapping. + +## Build and use + +Rebuild the extension from the repository root. This backend originally used +ABI 2; the current extension is ABI 3 after adding the +[persistent RNS backend](native-bfv-residue.md). An older binary raises a rebuild error. + +```bash +make -C src/xtrace_sdk/x_vec/crypto/bfv_cpu_ext PYTHON="$PWD/.venv/bin/python" +``` + +```python +import json +from xtrace_sdk.x_vec.crypto.bfv_client import BFVClient + +client = BFVClient(embed_len=512) +server = BFVClient(skip_key_gen=True, server_backend="native") +server.load_config(json.loads(client.stringify_config())) +server.load_stringified_keys(client.stringify_pk()) +assert server.keys is None + +query = [0, 1] * 256 +vectors = [query, [1 - bit for bit in query]] +encrypted_query = client.encrypt_vec_one(query) +encrypted_index = client.encrypt_vec_packed(vectors) +response = server.encode_hamming_server_packed(encrypted_query, encrypted_index, 2) +assert client.decode_hamming_client_packed(response, 2) == [0, 512] +``` + +The existing individual `encode_hamming_server` interface also uses the complete +native evaluator. Reuse a server across queries to reuse prepared public keys +and the full-tile mask. Loading keys or changing the configured layout resets +the native plan. `server_backend` remains a runtime option; the default is +`optimized`. This does not add BFV support to the production HTTP endpoints. + +## What profiling established + +The first instrumented full tile at N=8192 and dimension 512 spent about 35% of +native time in forward/inverse transforms and 30% in CRT reconstruction and +reduction modulo q. These measurements motivated both arithmetic changes and +moving the rest of the search into C++. The original Python profile could not +separate these costs inside a native call. + +The benchmark now supports `--native-profile`. After its timed searches, it runs +one additional warm search for each compiled backend and records exclusive +native phase durations and call counts. Nested timings are subtracted from their +parent category. Timers read the clock only during an opt-in profiling session. +Sessions are per-thread and restored when a call raises an exception. + +`other` includes Python work, binding overhead and native work outside named +scopes. `buffers` covers explicitly instrumented buffer setup. GMP allocations +remain included in the arithmetic operations that perform them; the profile +does not claim to isolate every allocator call. Use normal benchmark samples, +not instrumented runs, for speed comparisons. + +```bash +.venv/bin/python benchmarks/bfv_server.py --num-vectors 1024 \ + --backends rns,native --seal --repeats 4 --native-profile \ + --json-out /tmp/bfv-native-server-1024.json +``` + +Omit `--seal` when TenSEAL is unavailable. Native backends share exactly the +same encrypted input and must return identical ciphertexts. SEAL uses the same +plaintext vectors with its own parameters and format. The comparison is a local +server-call measurement, not an equal-security or identical-kernel comparison. +Native search times include ciphertext framing/buffer conversion, arithmetic +and compaction, but exclude outer MessagePack. SEAL includes its envelope and +binding temporary-file I/O. Key import, encryption, client decryption and network +latency are excluded from both. + +## Recorded results + +Measured on the same AMD Ryzen 7 5800X CPU with Python 3.12.3, GMP 6.3.0, +GCC 13.3.0 (`-O3`) and TenSEAL 0.3.16. Each vector has 512 bits. Both native +backends use N=8192, t=65537, the original 180-bit q, six 30-bit gadget digits, +and 50-bit response compaction. Each backend has one first search and three +subsequent searches; the warm column reports their median. + +| Vectors | Backend | First search (s) | Warm median (s) | +| ---: | --- | ---: | ---: | +| 32 | Previous `rns` | 0.4936 | 0.2602 | +| 32 | Complete C++ `native` | 0.5134 | 0.1639 | +| 32 | SEAL prototype | 0.0763 | 0.0741 | +| 1,024 | Previous `rns` | 8.2124 | 7.9266 | +| 1,024 | Complete C++ `native` | 5.6817 | 5.3426 | +| 1,024 | SEAL prototype | 2.3662 | 2.3592 | + +For 1,024 vectors, the complete C++ backend is **1.48 times faster** than the +previous RNS backend, reducing server time by **32.6%**. SEAL remains **2.26 +times faster** in this practical comparison. Native ciphertexts matched exactly +on every run. The 102,496-byte response, 471,250-byte query plus response and +26-bit minimum response noise budget are the same for both native backends. + +The additional whole-search profile explains where time changed: + +| Exclusive phase, 1,024 vectors | Previous `rns` (s) | Complete C++ `native` (s) | +| --- | ---: | ---: | +| Forward transforms | 1.562 | 1.565 | +| Inverse transforms | 0.759 | 0.557 | +| CRT reconstruction/conversion | 1.384 | 0.865 | +| Reduction modulo q | 0.486 | 0.491 | +| Pointwise work | 0.421 | 0.408 | +| Other: Python, binding and unscoped native work | 2.388 | 0.503 | + +The remaining phases are recorded in the JSON files. These are exclusive +instrumented durations, not additional speed benchmark samples. The biggest +reductions are in Python/binding work and CRT conversion; inverse transforms +also improve. Forward-transform and pointwise times are essentially unchanged +in this profile, so it does not establish an individual speedup for every new +primitive. The certified CRT fallback count is zero on these searches; the +boundary tests exercise it deliberately. + +The native plan prepares all 18 keys needed by this layout, while `rns` prepares +keys as operations request them (16 for 1,024 vectors, 11 for 32). This explains +the slightly higher first-use cost for the small workload. The native cache has +56,623,104 bytes of key coefficient arrays, 2,752,512 bytes of transform tables +and 655,360 bytes of prepared plaintext tables/mask. These are additional to the +loaded public key, exclude containers/CRT constants/temporary allocations, and +are not peak process memory. + +Raw samples, native profiles, parameters and source/binary hashes are in +`benchmarks/results/native_bfv_cpp_server_32.json` and +`benchmarks/results/native_bfv_cpp_server_1024.json`. The earlier benchmark +files are retained. + +## Changes inside the native server + +- `bfv_server.h` owns the public evaluation-key handles, prepared plaintext NTT + tables and transformed full-tile mask. A partial mask is local to a search. + The same mask transforms are reused for both ciphertext components. +- `packed_wire.h` imports/exports the original bit-packed polynomial format + directly in C++. It checks lengths and coefficient bounds. Python validates + the small framing header, without unpacking N Python integers per polynomial. +- A streaming binary merge tree retains O(log(padded dimension)) intermediate + ciphertexts. It parses each indexed tile once and keeps results in C++ through + compaction. Native calls release the GIL; plans are immutable and working + arrays belong to each call. +- The `native` arithmetic mode uses lazy NTT butterflies and reciprocal-based + Barrett reduction. The `rns` mode retains the earlier canonical butterflies + and general remainder operations for comparison. +- Key switching and prepared-mask multiplication compute their final result + modulo q using certified CRT conversion. Ciphertext multiplication still + reconstructs its full signed tensor product before BFV scale-and-round. + +These build on the [earlier SEAL source review](native-bfv-ntt.md). The +[SEAL evaluator](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/evaluator.cpp) +and [NTT routines](https://github.com/microsoft/SEAL/blob/v4.1.2/native/src/seal/util/ntt.cpp) +use specialized modular arithmetic and lazy transforms. Our implementations and +the CRT bounds below are independent. + +## Exact CRT conversion modulo q + +The auxiliary primes p_j still provide enough range to identify the signed +integer convolution, as described in [the original bounds](native-bfv-ntt.md). +Let M be their product, M_j = M/p_j, and r_j an output residue after inverse NTT. +Define: + +```text +y_j = r_j * inverse(M_j mod p_j) mod p_j +theta = sum(y_j / p_j) +k = floor(theta + 1/2) +signed result = sum(y_j * M_j) - k*M +``` + +We need only the last expression modulo q for key switching and mask products. +Precomputing `M_j mod q` and `M mod q` lets us form that residue without first +constructing the full integer modulo M. GMP still holds the final coefficients. + +The value k must be exact. With entirely integer arithmetic, compute: + +```text +F = sum(floor(2^64 * y_j / p_j)) +F/2^64 <= theta < (F + number_of_primes)/2^64 +``` + +If both ends of this interval round to the same integer, k is certified. If +they straddle a rounding boundary, use the original exact GMP CRT reconstruction +for that coefficient. No floating-point approximation or probabilistic +correctness assumption is used. Tests explicitly place integers on either side +of M/2 and verify that this fallback runs and agrees with the integer oracle. + +Ciphertext tensor products require the unreduced integer before multiplication +by t/q. They keep the original exact reconstruction and signed rounding rule; +reducing them modulo q early would be incorrect. + +## Native arithmetic bounds + +The fast forward transform keeps internal values below 2p and canonicalizes +its final output. The fast inverse keeps internal values below 4p and produces +canonical coefficients after inverse scaling. With p<2^60, these additions and +Shoup products fit the existing word bounds. Canonical forward outputs preserve +the key-switch accumulator proof: products are below 2^120, and batches of 256 +fit unsigned 128-bit accumulators. + +Barrett reduction stores `floor(2^128/p)`. The estimated quotient is at most one +below the true quotient, so a single correction gives the exact remainder for +any unsigned 128-bit input. The certified CRT fraction calculation specializes +this reciprocal calculation to `floor(y*2^64/p)`, with y