Skip to content

Add LFM2.5 2.6B support and speed up decode - #3

Merged
CerebralCoding merged 1 commit into
mainfrom
feat/lfm2-5-2-6b
Aug 11, 2026
Merged

Add LFM2.5 2.6B support and speed up decode#3
CerebralCoding merged 1 commit into
mainfrom
feat/lfm2-5-2-6b

Conversation

@CerebralCoding

Copy link
Copy Markdown
Owner

Summary

  • add strict runtime detection and native GGUF inference for the dense LFM2.5-2.6B model alongside LFM2.5-8B-A1B
  • derive layer schedules, tensor validation, cache sizing, FFN dimensions, and RoPE configuration from the detected model
  • keep decode workers hot between adjacent projections and reserve scheduler headroom for memory-bound matvecs
  • bump the project version to 0.2.0 and update the changelog, README, and architecture documentation

Performance

Using the official LFM2.5-2.6B Q4_K_M model and the 256-token benchmark:

  • decode: 68.33 → 90.33 tok/s (+32.2%)
  • prefill: 201.0 → 199.8 tok/s

Validation

  • ReleaseFast unit tests
  • ReleaseFast build
  • end-to-end tests with LFM2.5-2.6B and LFM2.5-8B-A1B
  • LFM2.5-2.6B tool-calling integration
  • benchmark continuation correctness check

@CerebralCoding
CerebralCoding marked this pull request as draft August 11, 2026 07:11
@CerebralCoding
CerebralCoding marked this pull request as ready for review August 11, 2026 07:11
@CerebralCoding
CerebralCoding merged commit a324cc8 into main Aug 11, 2026
3 of 4 checks passed
@CerebralCoding
CerebralCoding deleted the feat/lfm2-5-2-6b branch August 11, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant