parquet-to-blosc2 supports bi-directional conversion (--export converts Blosc2 back to Parquet). In contrast, b2nd-to-zarr only converts from Blosc2 .b2nd to Zarr.
Users with large existing Zarr datasets (v2 or v3) currently cannot convert them into .b2nd format using the CLI.
Suggested Work
- Extend
src/blosc2/cli/b2nd_to_zarr.py to support reverse conversion:
- Add a
--reverse / --export flag (or provide a secondary entrypoint script zarr-to-b2nd).
- Use
blosc2.ZarrNDSource or the zarr package to stream Zarr chunks into a target .b2nd NDArray in memory-bounded batches (--buffer-size).
- Map Zarr chunk dimensions to Blosc2 chunks, and compute appropriate Blosc2 block sizes.
- Preserve or translate compression parameters (codecs, shuffle filters) where possible.
- Add end-to-end roundtrip tests in
tests/test_cli_b2nd_to_zarr.py.
- Document the reverse conversion command in
doc/guides/b2nd_to_zarr.md.
- Analyze pros and cons of each format.
Bonus points: try to use a public Zarr dataset to offer a better overview in real situations.
parquet-to-blosc2supports bi-directional conversion (--exportconverts Blosc2 back to Parquet). In contrast,b2nd-to-zarronly converts from Blosc2.b2ndto Zarr.Users with large existing Zarr datasets (v2 or v3) currently cannot convert them into
.b2ndformat using the CLI.Suggested Work
src/blosc2/cli/b2nd_to_zarr.pyto support reverse conversion:--reverse/--exportflag (or provide a secondary entrypoint scriptzarr-to-b2nd).blosc2.ZarrNDSourceor thezarrpackage to stream Zarr chunks into a target.b2ndNDArray in memory-bounded batches (--buffer-size).tests/test_cli_b2nd_to_zarr.py.doc/guides/b2nd_to_zarr.md.Bonus points: try to use a public Zarr dataset to offer a better overview in real situations.