Skip to content

Add Apple Silicon Metal wheels - #21

Merged
andimarafioti merged 2 commits into
mainfrom
apple-silicon-metal-wheels
Sep 24, 2026
Merged

andimarafioti merged 2 commits into
mainfrom
apple-silicon-metal-wheels

Conversation

@andimarafioti

@andimarafioti andimarafioti commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Closes #19.

Apple Silicon users can install a macOS 14+ arm64 +metal wheel from the existing Hugging Face wheel channel, with libqwen and all ggml dylibs bundled and discoverable by QwenLibrary(). The PyPI release remains Linux CUDA 12.8. Publication follows merge by dispatching Publish Hugging Face Wheels; the PR check also exposes the wheel as an artifact.

  • Add a reusable macOS build/repair/check workflow with an explicit arm64 platform tag (including when building with universal2 Python), run it on PRs and from both validation and HF publishing, and document installation and local streaming tests.
  • Enable Metal with embedded shaders and Apple BLAS, relocate/version-map dylib dependencies, and restore valid arm64 signatures. Apply a shader-only workaround for the pinned ggml's invalid float-to-bfloat4 fill cast, which failed shader compilation on the test Mac; restore the native checkout after building.
  • Reject unverified native revisions and missing required symbols before any native parameter-buffer writes. Upstream exposes no safe ABI query, so this deliberately accepts only the verified pin. Compare every ctypes struct size/field offset against the pinned C header.
  • Fetch the exact pinned SHA explicitly in all build workflows: it is no longer reachable from upstream's default branch, and the previous clone/checkout sequence failed locally. The native pin and package release version are unchanged.

Validation on Apple M3 Pro, macOS 26.6.2, Python 3.12:

  • macOS 14 PR CI passed: native build, arm64 wheel repair, strict metadata checks, all 26 tests, and isolated installed loading.
  • 26 local tests passed, including native ABI layout checks and incompatible-library rejection.
  • Built qwentts_cpp_python-0.3.1+metal-py3-none-macosx_14_0_arm64.whl; delocate repair and twine check --strict passed. All six dylibs are arm64 and pass signature verification; external dependencies are Apple system libraries/frameworks only.
  • Installed into a fresh virtual environment using binary distributions. QwenLibrary().version() returned 7df559a (2026-07-17) without a library path, with the source/build/package-library trees moved away and library-path overrides unset.
  • From that isolated installation, streamed local 1.7B Base Q8_0 talker and Q8_0 tokenizer weights with GGML_BACKEND=MTL0: 7 finite, nonempty chunks, 2.56 seconds of non-silent mono 24 kHz PCM WAV. Native logs confirm Apple M3 Pro Metal execution. GPU access required running outside the tool sandbox.
  • HF index preparation, workflow YAML/TOML parsing, and whitespace checks passed. No build artifacts, wheels, or native libraries are committed.

The local GPU test is on macOS 26.6.2; the macOS 14 CI job verifies build/ABI/clean loading without model inference. The public HF wheel will be available after the publishing workflow runs on the merged change.

@andimarafioti
andimarafioti merged commit bf12a5f into main Sep 24, 2026
1 check passed
@andimarafioti
andimarafioti deleted the apple-silicon-metal-wheels branch September 24, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish an Apple Silicon Metal wheel

1 participant