Skip to content

build: prefer llama.cpp-mainline over sibling llama.cpp - #2

Closed
smanaton wants to merge 2 commits into
Apothic-AI:masterfrom
smanaton:prefer-mainline-llama
Closed

build: prefer llama.cpp-mainline over sibling llama.cpp#2
smanaton wants to merge 2 commits into
Apothic-AI:masterfrom
smanaton:prefer-mainline-llama

Conversation

@smanaton

@smanaton smanaton commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

When auto-detecting llama.cpp, prefer a sibling/checkout named llama.cpp-mainline over llama.cpp.

Some forks lag upstream ggml and have produced non-finite (NaN) logits on Q1_0 models. An explicit mainline directory name makes the intended tree obvious. WWAMA_LLAMA_CPP_DIR still overrides entirely.

Dependency

On Windows you still need the MSVC/CUDA build PR (or equivalent) to link successfully. Suggested merge order: windows-fixes first, then this.

Test plan

  • With this ordering + Windows build fixes, cargo build without WWAMA_LLAMA_CPP_DIR selected llama.cpp-mainline when present.
  • Finite baselines observed on Bonsai Q1_0 when building against mainline.

Notes

Fork staging: smanaton#2

smanaton added 2 commits July 19, 2026 10:34
The Apothic-AI fork (often checked out as a sibling llama.cpp) trails
upstream ggml and produces NaN logits on Q1_0 models. Prefer a
llama.cpp-mainline checkout when present so accidental builds against
the fork fail less silently. WWAMA_LLAMA_CPP_DIR still overrides entirely.
Copilot AI review requested due to automatic review settings July 19, 2026 18:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job was not started because the account is locked due to a billing issue.

@bitnom bitnom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address these before merge:

  1. build.rs:373-382 changes the candidate order but does not make Cargo reevaluate the resolver when llama.cpp-mainline is added after a fallback llama.cpp was selected. The script only emits rerun-if-changed for the selected directory (build.rs:23-24). Watch stable candidate parent/sentinel paths, or otherwise ensure a newly available preferred checkout invalidates the cached build.

  2. Please update the repository-layout section in README.md:80-91, which still says the lookup checks ../../cpp/llama.cpp first. The documented behavior now contradicts the build script.

  3. Add focused automated coverage for candidate ordering, explicit WWAMA_LLAMA_CPP_DIR precedence, and the no-candidate diagnostic. There are currently no CI checks on this PR.

The override semantics and candidate ordering are otherwise sensible, and this PR should remain dependent on the Windows link fixes being corrected.

@bitnom

bitnom commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Superseded by draft replacement PR #4: #4

The corrected implementation is on the upstream prefer-mainline-llama-fix feature branch, based on the corrected Windows feature branch. This fork branch cannot be updated from the maintainer account, so the original PR will be closed to keep the merge candidates unambiguous.

@bitnom

bitnom commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Closing as superseded by draft PR #4. The corrected work remains on the feature branches pending multi-platform validation.

@bitnom bitnom closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants