Skip to content

loader: validate GPU split sidecars before applying them - #288

Open
dandan20031031 wants to merge 1 commit into
Tiiny-AI:mainfrom
dandan20031031:fix/284-gpu-split-validation
Open

dandan20031031 wants to merge 1 commit into
Tiiny-AI:mainfrom
dandan20031031:fix/284-gpu-split-validation

Conversation

@dandan20031031

Copy link
Copy Markdown

Fixes #284.

The GPU split sidecar loader currently accepts negative and out-of-range bucket indices, installs them in model layers, and later uses them for host row selection. Validate the complete sidecar before publishing layer state or uploading bucket data, and keep an independent check immediately before row copies.

The validation covers tensor names, I32 vector layout, FFN row counts, file-contained tensor data, binary masks, bucket bounds, uniqueness, and agreement with the selected mask rows. Valid empty/full splits and unsorted bucket indices remain supported.

Related: #286 was closed without merging. This patch addresses the same underlying defect and additionally checks serialized sidecar data ranges, named tensor pairing, and mask/bucket consistency.

Validated locally on Windows, RTX 5080, CUDA 13.3.73, and MSVC 19.44:

  • The upstream loader accepted -1, the row-count upper bound, and INT32_MAX; the patched loader rejects all three without changing model-layer state.
  • 23 validation cases passed in both CUDA and CPU-only builds.
  • 15 serialized two-layer sidecar cases passed, including a bad second layer and truncated data.
  • Three row-copy cases passed: valid copies match the source bytes; negative and upper-bound indices throw before copying.
  • Existing CPU quantization and sampling tests passed.
  • Five Compute Sanitizer memcheck runs reported zero errors.

The serialized loader probe is retained with the local investigation materials. The repository regression test is model-free and can be run with ctest -R '^test-gpu-split$' after building its target. Full-model inference, Linux, multi-GPU operation, and general GGUF parser hardening are outside the validation scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OOB read / SEGV via unvalidated gpu_bucket row index in auto-loaded .generated.gpuidx sidecar

1 participant