Skip to content

Add serverless multi-collection benchmark mode - #173

Merged
generall merged 4 commits into
devfrom
feat/serverless-mode
Sep 2, 2026
Merged

Add serverless multi-collection benchmark mode#173
generall merged 4 commits into
devfrom
feat/serverless-mode

Conversation

@qdrant-cloud-bot

@qdrant-cloud-bot qdrant-cloud-bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements the BFB for serverless proposal: a dedicated bfb serverless mode that benchmarks Qdrant Serverless by spreading traffic across many per-tenant collections (instead of one shared collection).

bfb serverless upload --collection-prefix benchmark- --collections-count 100 \
  --distribution uniform --total-points 10M --config-file examples/serverless-upload.yaml

bfb serverless query --collection-prefix benchmark- --distribution zipf -n 10k

bfb serverless clear --collection-prefix benchmark-
  • New src/serverless/ module (CLI, client, registry, uniform/zipf routing, upload/clear/query)
  • Collections are created lazily on first upsert; registry reports preexisting / created / queryable
  • Reuses existing YAML upload shape + point generators; converts only the tenant-facing subset into QdrantServerless CollectionConfig
  • Bumps qdrant-client to dev tip with QdrantServerless
  • Query without a search YAML infers vector shape from an existing collection's config

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

Test plan

  • cargo test serverless (distribution allocate + config convert)
  • cargo clippy --all-targets -- -D warnings
  • bfb serverless --help / upload|clear|query --help
  • Live run against a serverless space (manual): upload → query → clear with QDRANT_API_KEY

Cursor Agent and others added 4 commits September 2, 2026 11:14
Cursor Agent
Add serverless multi-collection benchmark mode
1383e7b 
Introduce `bfb serverless {upload,clear,query}` for Qdrant Serverless:
lazily create per-tenant collections, route traffic with uniform/zipf
distributions, and drive the new QdrantServerless rust-client API.
Cursor Agent
Register serverless-upload in built-in examples catalog
48ae548 
The examples/ directory must match EXAMPLES; CI failed because the new
YAML was present on disk but not listed.
@generall @claude
Fix serverless mode review findings
2d32a4a 
- --rps paced requests one at a time; upload now uses an interval pacer
  with unbounded concurrency and query runs through the shared
  stats::process loop
- upload/query failures set the stop flag but exited 0; the first error
  is now returned
- upload throughput reported the requested count instead of acknowledged
  points; RPS/QPS series now come from the progress bar like regular search
- accept --file / --example (ConfigArgs) so the registered
  serverless-upload example is reachable; wire an optional search YAML
  into `serverless query` via ConfigSearchGenerator (dense, sparse,
  filters, IDF corpus, dataset query sets)
- derive a query template per dense and sparse vector when no YAML is
  given, skip empty collections, default to the shared 100k request count
- record upsert latency, honour --jsonl-updates, and populate
  BenchmarkResults so --json / --jsonl-searches / --jsonl-rps work
- per-slot create locks, verify preexisting collections against the
  YAML's vector shapes, lay point ids out contiguously across collections
  from --offset and cap totals by the dataset size
- make client::retry_with_clients generic and drop the serverless copy;
  build serverless clients from the shared get_config
- remove the unused queryable set and Notion references; document that
  the precision tier is left at the service default
- keep only the qdrant-client bump in Cargo.lock

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qdso7ZDKX1kUnn7VbPpVAE
@generall @claude
Add bfb serverless list
01615b6 
Lists collections (optionally filtered by --collection-prefix) with their
point counts and a total, so a space can be inspected before querying or
clearing it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qdso7ZDKX1kUnn7VbPpVAE
@generall
generall merged commit fb87f18 into dev Sep 2, 2026
3 checks passed
generall added a commit that referenced this pull request Sep 2, 2026
@qdrant-cloud-bot Cursor Agent
@generall @claude
Add serverless multi-collection benchmark mode (#173)
ffe02d1 
* Add serverless multi-collection benchmark mode

Introduce `bfb serverless {upload,clear,query}` for Qdrant Serverless:
lazily create per-tenant collections, route traffic with uniform/zipf
distributions, and drive the new QdrantServerless rust-client API.

* Register serverless-upload in built-in examples catalog

The examples/ directory must match EXAMPLES; CI failed because the new
YAML was present on disk but not listed.

* Fix serverless mode review findings

- --rps paced requests one at a time; upload now uses an interval pacer
  with unbounded concurrency and query runs through the shared
  stats::process loop
- upload/query failures set the stop flag but exited 0; the first error
  is now returned
- upload throughput reported the requested count instead of acknowledged
  points; RPS/QPS series now come from the progress bar like regular search
- accept --file / --example (ConfigArgs) so the registered
  serverless-upload example is reachable; wire an optional search YAML
  into `serverless query` via ConfigSearchGenerator (dense, sparse,
  filters, IDF corpus, dataset query sets)
- derive a query template per dense and sparse vector when no YAML is
  given, skip empty collections, default to the shared 100k request count
- record upsert latency, honour --jsonl-updates, and populate
  BenchmarkResults so --json / --jsonl-searches / --jsonl-rps work
- per-slot create locks, verify preexisting collections against the
  YAML's vector shapes, lay point ids out contiguously across collections
  from --offset and cap totals by the dataset size
- make client::retry_with_clients generic and drop the serverless copy;
  build serverless clients from the shared get_config
- remove the unused queryable set and Notion references; document that
  the precision tier is left at the service default
- keep only the qdrant-client bump in Cargo.lock

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qdso7ZDKX1kUnn7VbPpVAE

* Add `bfb serverless list`

Lists collections (optionally filtered by --collection-prefix) with their
point counts and a total, so a space can be inspected before querying or
clearing it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qdso7ZDKX1kUnn7VbPpVAE

---------

Co-authored-by: Cursor Agent <agent@qdrant.dev>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@generall generall generall approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@qdrant-cloud-bot @generall