Skip to content

[GUIDE] Walkthrough for b2nd-to-zarr CLI tool #714

Description

@FrancescAlted

b2nd_to_zarr is a robust CLI tool supporting Zarr v2 and v3, sharding (--sharded, --shards), custom chunk/codec mapping, memory buffer control (--buffer-size), and data integrity verification (--verify, --full-verify).

While parquet-to-blosc2 has a dedicated walkthrough guide in doc/guides/parquet_to_blosc2.rst, b2nd-to-zarr currently has no guide in doc/guides/ and is only mentioned in a single line within the installation docs.

Suggested Work

Create a new Markdown guide at doc/guides/b2nd_to_zarr.md (and add it to doc/guides/index.rst under Command line tools).

Cover the following steps with runnable shell and Python snippets:

  1. Prerequisites & Installation: Installing the [zarr] optional extra (pip install "blosc2[zarr]").
  2. Step 1 — Create a sample .b2nd array: A small Python snippet to generate a sample multidimensional array.
  3. Step 2 — Basic export to Zarr v3: Converting the file using b2nd-to-zarr sample.b2nd sample.zarr.
  4. Step 3 — Custom chunk layout & memory buffer: Overriding chunk shapes (--chunks) and controlling memory footprint with --buffer-size.
  5. Step 4 — Zarr v3 Sharding: Using --sharded to map Blosc2 chunks to Zarr shards and Blosc2 blocks to inner Zarr chunks.
  6. Step 5 — Codec and filter selection: Explicitly specifying codecs (--codec zstd, --clevel 5, --shuffle bitshuffle).
  7. Step 6 — Integrity verification: Checking data consistency with --verify and --full-verify.
  8. Cross-link with doc/guides/parquet_to_blosc2.rst and the Zarr source reference docs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions