scMultiBench

A systematic benchmark of single-cell multimodal integration, with multibench - a typed Python API that runs 40 integration methods across four scenarios (vertical, diagonal, mosaic, cross), scores them with scIB metrics, and draws the paper's figures.

Documentation and tutorials: https://dsichang.github.io/scMultiBench/

Quick start

pip install multibench-sc          # the API (import name: multibench)

For the tutorials (they also use the stored benchmark tables shipped in this repository), clone instead: git clone https://github.com/DSichang/scMultiBench.git && cd scMultiBench && pip install -e .

import multibench as mtb

mtb.list_methods()                 # the 40-method registry
mtb.method_info("Matilda")         # everything known about one method
mtb.scan("D11", "vertical")        # what can run on a dataset, and why not
res = mtb.run_all("D11", "vertical", out_dir="out/")   # run + score
res.plot()                         # the paper-style bubble panel

Running methods needs their conda environments (Linux). The package itself is ~2 MB - install only the environments you need:

multibench env doctor                              # what exists / is missing
multibench env install --methods Matilda --run     # one method (2-14 GB)
multibench env install --category vertical --run   # one category (45-101 GB)

The benchmark datasets are downloaded separately - see Get the data.

Try it without installing anything

The Colab quickstart installs the API, explores the registry, and reproduces the benchmark figures from the shipped result tables - entirely in the browser. The full published rankings are browsable in the interactive explorer.

Citation

Liu, Ding et al. Benchmarking single-cell multimodal data integrations. Nature Methods 22, 2449-2460 (2025). Every method you run is third-party software with its own paper - please cite it alongside the benchmark; mtb.method_info(name) points to each method's upstream repository and reference.