index.md

Platform Documentation

Documentation for database platforms supported by BenchBox.

Platform Guides

• Platform Selection Guide - Choose the right platform for your needs
• Quick Reference - Quick comparison and feature matrix
• Comparison Matrix - Detailed platform comparison
• Support Status - Registry support taxonomy and optional import diagnostics
• Deployment Modes - Platform deployment architecture (local, self-hosted, cloud)

DataFrame Platforms (Native API)

BenchBox supports benchmarking DataFrame libraries using their native APIs instead of SQL. This enables direct performance comparison between SQL and DataFrame paradigms on identical workloads.

• DataFrame Platforms Overview - Architecture, installation, and usage guide

Available DataFrame Platforms

Platform	CLI Name	Family	Status	Documentation
Polars	polars-df	Expression	Production-ready	Polars DataFrame
Pandas	pandas-df	Pandas	Production-ready	Pandas DataFrame
Dask	dask-df	Pandas	Production-ready	Dask DataFrame
cuDF	cudf-df	Pandas	Production-ready	cuDF DataFrame
PySpark	pyspark-df	Expression	Production-ready	PySpark DataFrame
LakeSail	lakesail-df	Expression	Production-ready	LakeSail DataFrame
DataFusion	datafusion-df	Expression	Production-ready	DataFusion DataFrame
Databricks	databricks-df	Expression	Production-ready	Databricks DataFrame

benchbox run --platform polars-df --benchmark tpch --scale 0.1
benchbox run --platform pandas-df --benchmark tpch --scale 0.1
benchbox run --platform dask-df --benchmark tpch --scale 0.1
benchbox run --platform cudf-df --benchmark tpch --scale 0.1
benchbox run --platform pyspark-df --benchmark tpch --scale 0.1
benchbox run --platform lakesail-df --benchmark tpch --scale 0.1
benchbox run --platform datafusion-df --benchmark tpch --scale 0.1
benchbox run --platform databricks-df --benchmark tpch --scale 0.1

benchbox run --platform polars --benchmark tpch --scale 0.1
benchbox run --platform polars-df --benchmark tpch --scale 0.1

Polars is the recommended starting point because it is fast. Pandas offers a familiar API. Dask is distributed. cuDF runs on GPUs and works on Linux only. PySpark fits the Spark ecosystem. LakeSail runs on Sail, a fast Spark-compatible engine. Databricks uses Databricks Connect.

The last two commands compare SQL and DataFrame execution on the same workload: --platform polars is SQL mode and --platform polars-df is DataFrame mode.

SQL Platforms

Core Local Databases

These platforms are included in the base BenchBox installation with no additional dependencies:

• DuckDB - Embedded analytical database (default local platform)
• SQLite - Embedded row-store database for lightweight testing

Local/Embedded Analytics Engines

• Apache DataFusion - Rust-based in-memory query engine
• ClickHouse Local - Embedded ClickHouse via chDB (zero-config)
• ClickHouse Server - Self-hosted ClickHouse via clickhouse-driver
• Polars - High-performance DataFrame library with SQL support

Traditional Relational Databases

• PostgreSQL - Open-source relational database with TimescaleDB support
• CedarDB - High-performance HTAP engine over PostgreSQL wire protocol (formerly Umbra)

PostgreSQL Extensions

• pg_duckdb - DuckDB-powered Postgres extension for vectorized OLAP
• pg_mooncake - Columnar storage for Postgres with Iceberg and Delta Lake support
• paradedb - Hybrid search and analytics Postgres extension (BM25 via pg_analytics)
• citus - Distributed Postgres extension with sharded tables

Self-Hosted OLAP Databases

• StarRocks - MPP columnar OLAP database (MySQL protocol + Stream Load)
• Apache Doris - MPP real-time analytical database (MySQL protocol + Stream Load)
• SingleStore - Distributed SQL with columnstore analytics (MySQL protocol + LOAD DATA)
• QuestDB - Time-series database (PostgreSQL wire protocol + REST API)

Distributed SQL Engines

• PrestoDB - Distributed SQL query engine (Facebook fork)
• Trino - Distributed SQL query engine (community fork, formerly PrestoSQL)
• Apache Spark - Unified analytics engine for large-scale data processing
• LakeSail Sail - High-performance Spark-compatible engine (Spark Connect)
• Apache Gluten + Velox - Spark plugin that offloads physical operators to a vectorized C++ engine (Linux-only, Docker on macOS/Windows)

Cloud Data Warehouses

• ClickHouse Cloud - Managed ClickHouse service (inherits ClickHouse dialect)
• Snowflake - Cloud-native data platform (AWS, Azure, GCP)
• Databricks - Lakehouse platform with Unity Catalog support
• Onehouse Quanton - Serverless Spark with multi-format support (Hudi, Iceberg, Delta)
• Google BigQuery - Serverless data warehouse
• Amazon Redshift - Columnar data warehouse
• MotherDuck - Serverless DuckDB cloud (inherits DuckDB dialect)
• DuckLake - DuckDB lakehouse format (Parquet data + SQL catalog; inherits DuckDB dialect)
• Starburst - Managed Trino / Starburst Galaxy (inherits Trino dialect)
• Amazon Athena - Serverless query service for S3
• Firebolt - High-performance cloud analytics (Core + Cloud modes)
• Databend - Cloud-native OLAP warehouse (Snowflake-compatible SQL, S3/MinIO storage)
• Azure Synapse Analytics - Microsoft cloud analytics platform
• Microsoft Fabric Warehouse - T-SQL warehouse with OneLake-native storage
• Microsoft Fabric Lakehouse SQL - Read-only SQL endpoint over Fabric lakehouses
• TimescaleDB / TigerData - TimescaleDB with managed TigerData cloud mode (timescaledb:cloud)

GPU-Accelerated Platforms

• CUDF - NVIDIA RAPIDS GPU-accelerated DataFrames

Platform Categories

By Installation Complexity

Zero Config (included in base install):

• DuckDB, SQLite

Single Extra (one pip install command):

• DataFusion, Polars, PostgreSQL, ClickHouse

Cloud SDK Required (authentication setup needed):

• Databricks SQL, Databend, BigQuery, Redshift, Snowflake, Amazon Athena, Firebolt, Azure Synapse Analytics

Infrastructure Required (external cluster needed):

• Trino, Presto, Spark, LakeSail, Gluten+Velox (Linux/Docker), StarRocks, Doris, QuestDB, ClickHouse (server mode)

By Use Case

Local Development & Testing:

• DuckDB (recommended), SQLite, DataFusion, Polars

Production Benchmarking:

• Databricks, Snowflake, BigQuery, Redshift

Self-Hosted Analytics:

• ClickHouse, StarRocks, Doris, QuestDB, PostgreSQL, Trino, Presto, Spark, LakeSail, Gluten+Velox

GPU Workloads:

• CUDF (NVIDIA GPUs required)

Quick Start by Platform

Local Platforms (No Setup Required)

benchbox run --platform duckdb --benchmark tpch --scale 0.01

benchbox run --platform sqlite --benchmark tpch --scale 0.01

DuckDB is the default and is included in the base install. SQLite is also included in the base install.

Cloud Platforms (Credentials Required)

benchbox run --platform databricks --benchmark tpch --scale 1.0

benchbox run --platform bigquery --benchmark tpch --scale 1.0

benchbox run --platform snowflake --benchmark tpch --scale 1.0

Databricks requires DATABRICKS_TOKEN and DATABRICKS_HOST. BigQuery requires GOOGLE_APPLICATION_CREDENTIALS. Snowflake requires SNOWFLAKE_USER, SNOWFLAKE_PASSWORD and SNOWFLAKE_ACCOUNT.

Future Platforms

• Development Roadmap - Planned platform and benchmark additions

Related Documentation

• Getting Started - Quick start guide
• Cloud Storage Guide - Cloud storage configuration for staging data
• Platform Configuration - Detailed configuration options

:maxdepth: 1

platform-selection-guide
quick-reference
comparison-matrix
support-status
deployment-modes
dataframe
duckdb
sqlite
polars
pandas-dataframe
modin-dataframe
dask-dataframe
cudf
pyspark-dataframe
datafusion-dataframe
databricks-dataframe
spark
lakesail
velox
velox_jar_setup
starrocks
doris
singlestore
questdb
databend
cedardb
clickhouse-local-mode
clickhouse-server
clickhouse-cloud
clickhouse-migration
postgresql
pg_duckdb
pg_mooncake
paradedb
citus
presto
trino
snowflake
databricks
bigquery
redshift
motherduck
ducklake
starburst
quanton
athena
athena-spark
firebolt
microsoft-fabric
fabric-lakehouse
azure-platforms
datafusion
influxdb
timescaledb
aws-glue
gcp-dataproc
dataproc-serverless
emr-serverless
fabric-spark
synapse-spark
snowpark-connect