Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 44 additions & 5 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Build, test and publish lance-csr wheels: Linux x86-64 (manylinux_2_28) and native
# Apple Silicon (macOS arm64), plus an sdist for every other platform.
# Build, test and publish lance-csr wheels: Linux x86-64 (manylinux_2_28), native
# Apple Silicon (macOS arm64) and Windows x86-64, plus an sdist for every other platform.
#
# Pull requests that change how wheels are built (this workflow, the wheel check, the
# release build profile) build and test them without publishing. A release builds and tests
Expand Down Expand Up @@ -135,15 +135,53 @@ jobs:
path: python/dist/*.whl
if-no-files-found: error

build-windows:
runs-on: windows-2022
timeout-minutes: 90
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- uses: astral-sh/setup-uv@v4

- name: Install build dependencies
run: choco install protoc --version=29.3 -y --no-progress

- name: Build Windows x86-64 wheel
uses: PyO3/maturin-action@04ac600d27cdf7a9a280dadf7147097c42b757ad # v1
with:
command: build
working-directory: python
target: x86_64-pc-windows-msvc
# sccache's Windows server dropped the connection mid-way through compiling lance.
sccache: "false"
args: --release --locked --strip --out dist

- name: Inspect wheel
working-directory: python
shell: bash
run: |
uvx twine check dist/*.whl
sha256sum dist/*.whl

- uses: actions/upload-artifact@v4
with:
name: python-dist-windows-x64
path: python/dist/*.whl
if-no-files-found: error

test:
# The wheels are abi3 (CPython 3.12+), so each platform's single wheel is tested on
# every supported Python, in a clean environment with Rust removed from PATH: a wheel
# that silently needed compiling would fail here.
needs: [build-linux, build-macos]
needs: [build-linux, build-macos, build-windows]
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, macos-15]
os: [ubuntu-24.04, macos-15, windows-2022]
python: ["3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
Expand Down Expand Up @@ -171,6 +209,7 @@ jobs:
case "$RUNNER_OS" in
Linux) wheel=$(ls "$PWD"/dist/*manylinux*.whl) ;;
macOS) wheel=$(ls "$PWD"/dist/*macosx*.whl) ;;
Windows) wheel=$(cygpath -w "$(ls "$PWD"/dist/*win_amd64*.whl)") ;;
esac
# A fresh environment with the wheel and test tools; --no-build refuses any sdist.
run=(uv run --isolated --no-project --no-build --python ${{ matrix.python }}
Expand All @@ -181,7 +220,7 @@ jobs:
"${run[@]}" python -I -m pytest "$RUNNER_TEMP/tests" -q -p no:cacheprovider

publish:
needs: [build-linux, build-macos, test]
needs: [build-linux, build-macos, build-windows, test]
if: >
github.event_name == 'release' ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.mode == 'release')
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ case needs distributed graph queries or graph algorithms, that's the kind of eng
uv add lance-csr
```

Prebuilt wheels cover Linux x86-64 and native Apple Silicon (CPython 3.12+); no Rust toolchain is needed. From
Prebuilt wheels cover Linux x86-64, native Apple Silicon and Windows x86-64 (CPython 3.12+); no Rust toolchain is needed. From
Rust, run `cargo add lance-csr` and see the [Rust guide](https://lance-csr.hdclabs.ai/rust/). To build from
source, see [`python/DEVELOPMENT.md`](python/DEVELOPMENT.md).

Expand Down
3 changes: 2 additions & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A release publishes the Python package to [PyPI](https://pypi.org/project/lance-
`lance-csr` crate to [crates.io](https://crates.io/crates/lance-csr). PyPI publishing is
automated from a GitHub release, and the crates follow once PyPI has the new version.

The whole release takes about 50 minutes: the macOS wheel build, then the crates. Don't add steps that build the
The whole release takes about 65 minutes: the Windows wheel build, then the crates. Don't add steps that build the
release wheels a second time.

## Before you start
Expand Down Expand Up @@ -66,6 +66,7 @@ Publishing it starts `.github/workflows/python-publish.yml`:
| --- | ---: | --- |
| `build-linux` | about 22 min | manylinux_2_28 x86-64 wheel and the sdist |
| `build-macos` | about 35 min | macOS arm64 wheel, built natively on `macos-15` |
| `build-windows` | about 52 min | Windows x86-64 wheel, built natively on `windows-2022` |
| `test` | under 1 min each | each wheel on Python 3.12, 3.13 and 3.14 |
| `publish` | seconds | checks the tag matches `python/pyproject.toml`, then uploads through PyPI trusted publishing |
| `publish-crates` | about 10 to 15 min | after `publish`: packages and verifies `lance-csr`, then uploads it through crates.io trusted publishing |
Expand Down
3 changes: 3 additions & 0 deletions python/.cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,8 @@ rustflags = [
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "target-cpu=haswell", "-C", "target-feature=+avx2,+fma,+f16c"]

[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-cpu=haswell", "-C", "target-feature=+avx2,+fma,+f16c"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "target-cpu=apple-m1", "-C", "target-feature=+neon,+fp16"]
5 changes: 3 additions & 2 deletions python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@ Or run a script with it, without a project:
uv run --with lance-csr python your_script.py
```

Prebuilt wheels are published for Linux x86-64 (glibc 2.28 or newer: RHEL 8, Ubuntu 20.04 and later)
and native Apple Silicon (macOS 11 or newer), for CPython 3.12 and later. Installing from a wheel needs
Prebuilt wheels are published for Linux x86-64 (glibc 2.28 or newer: RHEL 8, Ubuntu 20.04 and later),
native Apple Silicon (macOS 11 or newer) and Windows x86-64, for CPython 3.12 and later. The x86-64
wheels need a CPU with AVX2 (Intel Haswell, AMD Excavator or newer). Installing from a wheel needs
no Rust toolchain; to make sure uv never falls back to compiling from source, use:

```bash
Expand Down
1 change: 1 addition & 0 deletions python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ classifiers = [
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
"Operating System :: MacOS",
"Operating System :: Microsoft :: Windows",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
Expand Down
Loading