Databend Platform

Databend is a cloud-native, Rust-based data warehouse with Snowflake-compatible SQL, compute/storage separation, and object storage backends (S3, GCS, Azure Blob, MinIO). BenchBox connects via the databend-driver Python package (DB-API 2.0 compliant) and uses the Snowflake dialect as a SQLGlot translation proxy for SQL compatibility.

Databend supports two deployment modes: Databend Cloud (managed service) and self-hosted (user-managed cluster with object storage). Its vectorized query engine, written in Rust, is optimized for analytical workloads with automatic micro-partitioning and clustering key support.

Features

Cloud-native architecture - Compute/storage separation on object storage (S3, MinIO, GCS, Azure Blob)
Snowflake-compatible SQL - Uses Snowflake dialect as SQLGlot translation proxy (~100% compatibility)
Full TPC-H support - All 22 queries with row count validation
Full TPC-DS support - All 99 queries with row count validation
Vectorized Rust engine - High-performance analytical query execution
DB-API 2.0 driver - Standard Python database connectivity via databend-driver
Clustering keys - CLUSTER BY clause for optimizing data layout
Automatic statistics - No explicit ANALYZE needed; statistics collected by storage engine
Result cache control - Configurable query result cache (disabled by default for benchmarking)
Dual deployment - Databend Cloud (managed) or self-hosted with MinIO/S3

Quick Start

Install the databend-driver dependency directly or through the Databend extra, then configure the connection for Databend Cloud and run TPC-H:

uv add databend-driver

uv add benchbox --extra databend

export DATABEND_HOST=tenant--warehouse.gw.databend.com
export DATABEND_USER=benchbox
export DATABEND_PASSWORD=your_password

benchbox run --platform databend --benchmark tpch --scale 0.01

Self-Hosted Quick Start

The Docker image needs MinIO or S3-compatible storage. Select the self-hosted deployment with databend:self-hosted. It connects over plain HTTP on port 8000 unless you set ssl or port, and the result is recorded as the self-hosted deployment, separate from Databend Cloud runs:

docker run -p 8000:8000 datafuselabs/databend:latest

export DATABEND_HOST=localhost

benchbox run --platform databend:self-hosted --benchmark tpch --scale 0.01

DSN-Based Connection

benchbox run --platform databend --benchmark tpch --scale 0.01 \
    --platform-option dsn=databend+http://benchbox:benchbox@localhost:8000/benchbox?sslmode=disable

Platform Options

Option	Environment Variable	Default	Description
`host`	`DATABEND_HOST`	-	Databend host (required unless DSN is set)
`port`	`DATABEND_PORT`	`443` (SSL) / `8000` (no SSL)	Connection port
`username`	`DATABEND_USER`	`benchbox`	Database username
`password`	`DATABEND_PASSWORD`	-	Database password
`database`	`DATABEND_DATABASE`	`benchbox`	Target database name
`dsn`	`DATABEND_DSN`	-	Full DSN string (overrides individual params)
`warehouse`	`DATABEND_WAREHOUSE`	-	Databend Cloud warehouse name
`ssl`	-	`true` for `databend` and `databend:cloud`, `false` for `databend:self-hosted`	SSL/TLS for connections
`disable_result_cache`	-	`true`	Disable query result cache for benchmarking

DSN Format

The DSN (Data Source Name) follows the format:

databend+https://user:password@host:port/database?warehouse=name
databend+http://user:password@host:port/database?sslmode=disable

databend+https:// is used when SSL is enabled (default for cloud)
databend+http:// is used when SSL is disabled (typical for self-hosted) and should include sslmode=disable
Special characters in username/password are automatically URL-encoded

Data Loading

BenchBox loads data into Databend using batch INSERT statements via the databend-driver Python client. The adapter handles both TPC pipe-delimited (.tbl) and standard CSV formats automatically.

Loading Process

Database creation - CREATE DATABASE IF NOT EXISTS ensures target database exists
Schema creation - Tables created with Snowflake-compatible DDL, optimized for Databend
Type conversion - CHAR(n) converted to VARCHAR(n) (Databend preference); constraints removed
Batch inserts - Data loaded in batches of 500 rows using INSERT INTO ... VALUES statements
Constraint handling - Primary keys and foreign keys are removed (Databend does not enforce them)

Type Mappings

Source Type	Databend Type
`CHAR(n)`	`VARCHAR(n)`
`INTEGER`	`INTEGER` (unchanged)
`VARCHAR(n)`	`VARCHAR(n)` (unchanged)
`DECIMAL(p,s)`	`DECIMAL(p,s)` (unchanged)
`DATE`	`DATE` (unchanged)
`DOUBLE`	`DOUBLE` (unchanged)

Large-Scale Loading

For datasets larger than SF 10, Databend supports high-throughput loading via:

COPY INTO from staged S3/MinIO files (for cloud and self-hosted with object storage)
Streaming Load via HTTP API for parallel ingestion

The current adapter uses INSERT batching for broad compatibility across both deployment modes.

Usage Examples

Basic Benchmarks

benchbox run --platform databend --benchmark tpch --scale 1.0

benchbox run --platform databend --benchmark tpcds --scale 10.0

benchbox run --platform databend --benchmark tpch --queries Q1,Q6,Q17

Databend Cloud Configuration

export DATABEND_HOST=tenant--warehouse.gw.databend.com
export DATABEND_USER=benchbox
export DATABEND_PASSWORD=your_cloud_password
export DATABEND_WAREHOUSE=my_warehouse

benchbox run --platform databend --benchmark tpch --scale 10.0

Self-Hosted Configuration

Select the self-hosted deployment. SSL is off and the port is 8000 by default:

export DATABEND_HOST=localhost

benchbox run --platform databend:self-hosted --benchmark tpch --scale 1.0

Add --platform-option ssl=true to a self-hosted run that terminates TLS.

Custom Database Name

benchbox run --platform databend --benchmark tpch --scale 1.0 \
    --platform-option database=my_benchmarks

Dry Run (Preview)

benchbox run --platform databend --benchmark tpch --scale 1.0 --dry-run ./preview

Architecture

Adapter Structure

The Databend adapter is a single-file adapter with all functionality in one class:

Module	Class	Responsibility
`adapter.py`	`DatabendAdapter`	Connection, schema, loading, queries, tuning
`__init__.py`	`_build_databend_config`	Config builder with credential loading

Connection Model

BenchBox CLI
    |
    v
DatabendAdapter
    |
    +-- databend-driver (DB-API 2.0)
    |       - BlockingDatabendClient
    |       - DSN-based connection (databend+http:// or databend+https://)
    |
    +-- Databend Cloud (HTTPS, port 443)
    |       - tenant--warehouse.gw.databend.com
    |       - Warehouse-scoped compute
    |
    +-- Self-hosted (HTTP, port 8000)
            - Direct connection to Databend server
            - Object storage backend (MinIO/S3)

SQL Translation Strategy

Databend claims ~100% Snowflake SQL compatibility. BenchBox leverages this by:

Translating benchmark SQL from DuckDB dialect to Snowflake dialect via SQLGlot
Sending Snowflake-compatible SQL directly to Databend
Applying edge-case optimizations in _optimize_table_definition():
- CHAR(n) to VARCHAR(n) conversion
- Primary key constraint removal
- Foreign key constraint removal

Platform Information

At runtime, BenchBox captures platform metadata:

{
    "platform_type": "databend",
    "platform_name": "Databend",
    "configuration": {
        "host": "tenant--warehouse.gw.databend.com",
        "database": "benchbox_tpch_sf1"
    },
    "warehouse": "my_warehouse",
    "platform_version": "v1.x.x",
    "client_library_version": "0.x.x"
}

Tuning and Optimization

Automatic Benchmark Configuration

The adapter automatically applies optimizations when running benchmarks:

Query result cache: Disabled by default (enable_query_result_cache = 0) for accurate timing
Vectorized engine: Pre-optimized for analytical workloads (no additional tuning needed)
Automatic statistics: Databend collects statistics via its storage engine; no explicit ANALYZE required

Supported Tuning Types

Tuning Type	Support	Notes
Clustering	Yes	`CLUSTER BY` clause for data layout optimization
Partitioning	No	Automatic micro-partitioning (not user-configurable)
Distribution	No	Managed by storage engine

Clustering Keys

Databend supports CLUSTER BY to optimize data layout for frequently queried columns, similar to Snowflake clustering keys. BenchBox applies this automatically via the tuning configuration:

ALTER TABLE lineitem CLUSTER BY (l_shipdate, l_orderkey)

Clustering keys can be specified at table creation time (via generate_tuning_clause) or applied post-creation (via apply_table_tunings).

Custom Platform Settings

benchbox run --platform databend --benchmark tpch --scale 10.0 \
    --tuning tuned

Troubleshooting

Connection Refused

Error: Databend configuration is incomplete

Solutions:

Verify Databend is running and accessible on the configured host and port
For cloud: Set DATABEND_HOST=tenant--warehouse.gw.databend.com
For self-hosted: Set DATABEND_HOST=localhost and DATABEND_PORT=8000
Provide either DSN, individual params, or environment variables

Missing databend-driver Dependency

Error: Missing dependencies for databend platform: databend-driver

Solutions:

Install databend-driver: uv add databend-driver
Or install the Databend extra: uv add benchbox --extra databend
Minimum version required: >=0.28.0

SSL/TLS Errors

Error: SSL handshake failed

Solutions:

For self-hosted without TLS: --platform-option ssl=false
For cloud: SSL is required; verify certificate chain
Check that the port matches the SSL setting (443 for SSL, 8000 for non-SSL)

Schema Creation Failures

Error: Schema creation failed

Solutions:

Check that the user has CREATE TABLE and CREATE DATABASE permissions
Verify the database exists or the user has CREATE DATABASE privilege
If tables already exist, the adapter will automatically drop and recreate them
Review Databend query logs for detailed error messages

Slow Data Loading

Solutions:

For large datasets (SF 10+), consider using Databend's native COPY INTO from staged files
Ensure adequate network bandwidth between BenchBox host and Databend
For self-hosted: verify object storage (MinIO/S3) performance
Consider running BenchBox on a machine co-located with the Databend cluster

Query Result Cache Interference

Warning: Query results may be cached

Solutions:

Verify disable_result_cache is true (default): --platform-option disable_result_cache=true
The adapter sets enable_query_result_cache = 0 at session level
For Databend Cloud: check that warehouse-level caching is not interfering

Navigation Menu

Repository navigation

databend.md

databend.md

Databend Platform

Features

Quick Start

Self-Hosted Quick Start

DSN-Based Connection

Platform Options

DSN Format

Data Loading

Loading Process

Type Mappings

Large-Scale Loading

Usage Examples

Basic Benchmarks

Databend Cloud Configuration

Self-Hosted Configuration

Custom Database Name

Dry Run (Preview)

Architecture

Adapter Structure

Connection Model

SQL Translation Strategy

Platform Information

Tuning and Optimization

Automatic Benchmark Configuration

Supported Tuning Types

Clustering Keys

Custom Platform Settings

Troubleshooting

Connection Refused

Missing databend-driver Dependency

SSL/TLS Errors

Schema Creation Failures

Slow Data Loading

Query Result Cache Interference

See Also

FilesExpand file tree

databend.md

Latest commit

History

databend.md

File metadata and controls

Databend Platform

Features

Quick Start

Self-Hosted Quick Start

DSN-Based Connection

Platform Options

DSN Format

Data Loading

Loading Process

Type Mappings

Large-Scale Loading

Usage Examples

Basic Benchmarks

Databend Cloud Configuration

Self-Hosted Configuration

Custom Database Name

Dry Run (Preview)

Architecture

Adapter Structure

Connection Model

SQL Translation Strategy

Platform Information

Tuning and Optimization

Automatic Benchmark Configuration

Supported Tuning Types

Clustering Keys

Custom Platform Settings

Troubleshooting

Connection Refused

Missing databend-driver Dependency

SSL/TLS Errors

Schema Creation Failures

Slow Data Loading

Query Result Cache Interference

See Also