Skip to content

Metal backend: stage compiled .mlmodelc next to source .mlpackage #1274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhen-ke wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Metal backend: stage compiled .mlmodelc next to source .mlpackage #1274

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
91bc44e
Metal backend: stage compiled .mlmodelc next to source .mlpackage
zhen-ke Oct 27, 2026
5f6aacf
Metal backend: do not delete compiled bundles on failure
zhen-ke Oct 27, 2026
ab94e23
Metal backend: update comments per review
zhen-ke Oct 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion cpp/neuralnet/metalbackend.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,8 +375,26 @@ public func createCoreMLComputeHandle(
printError("Metal backend \(serverThreadIdx): Compiling model...")
let compiledURL = try MLModel.compileModel(at: mlpackagePath)

// compileModel(at:) writes the .mlmodelc into NSTemporaryDirectory(), which
// ignores TMPDIR. Move it next to the .mlpackage the C++ side passed in
// (temp_directory_path(), which honors TMPDIR) before loading, so the bundle
// shares that directory's lifetime. Derive the destination from that path
// rather than resolving the temp directory again here.
// A failed move is not fatal: load from where compileModel left the bundle.
// Do not delete the compiled bundle on either path: MLModel may read it lazily.
let stagedURL = mlpackagePath.deletingLastPathComponent()
.appendingPathComponent(compiledURL.lastPathComponent)
let modelURL: URL
do {
try FileManager.default.moveItem(at: compiledURL, to: stagedURL)
modelURL = stagedURL
} catch {
printError("Metal backend \(serverThreadIdx): Warning: Failed to move compiled model to \(stagedURL.path), loading it from \(compiledURL.path): \(error)")
modelURL = compiledURL
}

printError("Metal backend \(serverThreadIdx): Loading compiled model...")
let model = try MLModel(contentsOf: compiledURL, configuration: config)
let model = try MLModel(contentsOf: modelURL, configuration: config)

printError("Metal backend \(serverThreadIdx): Model loaded successfully, \(context.nnXLen)x\(context.nnYLen)")

Expand Down