build: prefer llama.cpp-mainline over sibling llama.cpp - #2
Conversation
The Apothic-AI fork (often checked out as a sibling llama.cpp) trails upstream ggml and produces NaN logits on Q1_0 models. Prefer a llama.cpp-mainline checkout when present so accidental builds against the fork fail less silently. WWAMA_LLAMA_CPP_DIR still overrides entirely.
bitnom
left a comment
There was a problem hiding this comment.
Please address these before merge:
-
build.rs:373-382changes the candidate order but does not make Cargo reevaluate the resolver whenllama.cpp-mainlineis added after a fallbackllama.cppwas selected. The script only emitsrerun-if-changedfor the selected directory (build.rs:23-24). Watch stable candidate parent/sentinel paths, or otherwise ensure a newly available preferred checkout invalidates the cached build. -
Please update the repository-layout section in
README.md:80-91, which still says the lookup checks../../cpp/llama.cppfirst. The documented behavior now contradicts the build script. -
Add focused automated coverage for candidate ordering, explicit
WWAMA_LLAMA_CPP_DIRprecedence, and the no-candidate diagnostic. There are currently no CI checks on this PR.
The override semantics and candidate ordering are otherwise sensible, and this PR should remain dependent on the Windows link fixes being corrected.
|
Superseded by draft replacement PR #4: #4 The corrected implementation is on the upstream |
|
Closing as superseded by draft PR #4. The corrected work remains on the feature branches pending multi-platform validation. |
Summary
When auto-detecting llama.cpp, prefer a sibling/checkout named
llama.cpp-mainlineoverllama.cpp.Some forks lag upstream ggml and have produced non-finite (NaN) logits on Q1_0 models. An explicit mainline directory name makes the intended tree obvious.
WWAMA_LLAMA_CPP_DIRstill overrides entirely.Dependency
On Windows you still need the MSVC/CUDA build PR (or equivalent) to link successfully. Suggested merge order: windows-fixes first, then this.
Test plan
cargo buildwithoutWWAMA_LLAMA_CPP_DIRselectedllama.cpp-mainlinewhen present.Notes
Fork staging: smanaton#2