Saturday's release burst is the longest one-day tag sequence llama.cpp has cut in 2026 and reads as a snapshot of the project's role in the open-source inference stack. Georgi Gerganov's maintainer team and roughly two dozen weekend contributors moved 18 signed pre-release tags from b10688 through b10705 into the release feed between 00:11 UTC and 23:15 UTC, each carrying 29 platform binaries: macOS arm64 and x64, iOS as an XCFramework, Ubuntu x64 with CPU / Vulkan / ROCm / OpenVINO / SYCL variants, Windows with CPU / CUDA 12 and 13 / Vulkan / ROCm / OpenVINO / SYCL, Android arm64, openEuler, and the built-in web UI. The repository crossed 126,000 GitHub stars and 22,500 forks during the sequence and has 870 open issues.
"prompt processing 2334 → 2600 t/s" — b10704 release notes, RTX PRO 6000, Qwen3.8-Flash-Next, 512 experts, 55K context
The Qwen3.8-Flash-Next fast path
The highest-signal tag is b10704 at 22:54 UTC. The prior mm_ids_helper fast path in the CUDA MoE routing kernel required warp_size % n_expert_used == 0 with a hardcoded exception for six experts, which meant Qwen3.8-Flash-Next's n_expert_used=10 configuration fell back to the slow path. b10704 generalizes the guard to the next power of two, letting any expert-count configuration hit the fast helper. The measured effect on an RTX PRO 6000 running Qwen3.8-Flash-Next at 512 experts and 55K context is prompt processing lifting from 2,334 to 2,600 tokens per second, a roughly 11 percent gain. Token-generation throughput is unchanged. The change lands three days after Alibaba pushed the Qwen3.8-Flash-Next open-weights preview and effectively raises the throughput floor for anyone running the model on consumer or prosumer Nvidia hardware.
Apple Silicon, one chip per tag
Three separate tags land Metal flash-attention vector tunings. b10688 covers the base M2, b10696 the M3 Pro and b10697 the M3 Ultra. Each tag adds hand-tuned parameter selections for the fa-vec kernel path on that specific SoC family. b10691 is the more consequential Metal fix in the day's set: it patches a null-pipeline crash when F16 src1 mul_mat and mul_mat_id operations dispatch on a Metal encoder without a matching kernel available. The prior behavior was a segfault inside encoder_set_pipeline; b10691 fails closed with a supports_op check and aborts on nil pipeline rather than crashing. b10698 clears an unrelated Apple RDMA teardown error spew, and b10694 restores RPC compatibility on macOS releases predating the RDMA integration.
"metal: fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id" — b10691 commit title
AMD RDNA3 and RDNA4
Two HIP tags land in the same evening cluster. b10702 at 21:20 UTC optimizes the Q2_0 dot-product path for gfx1201, AMD's RDNA 4 architecture, using native AMD GCN permutation operations and reworked MMQ tile-index handling. b10703 at 22:31 UTC retunes the RDNA 3 MMQ config. Both changes materially improve local-inference throughput on Radeon RX 9000 and RX 7000-class parts, closing the gap with Nvidia hardware for anyone running quantized MoE models at home.
Qualcomm Hexagon and the on-demand session model
b10693 at 12:43 UTC adds "support for device discovery and create sessions on demand" for the Hexagon NPU backend, contributed by Marco Colombo, Alexander Lu and Ehsan Bateni at Qualcomm. The prior Hexagon path assumed a fixed session at init; the new code enumerates available NPU cores at runtime, lazily allocates and cleans up sessions, and rejects non-existent devices early. The change matters for the emerging class of Snapdragon X and Snapdragon 8 Gen 4 laptops and phones running llama.cpp locally against the NPU rather than the CPU or GPU.
DFlash2 speculative decoding restored on NVFP4
b10701 at 18:56 UTC is a functional fix rather than a performance change. DFlash2, llama.cpp's second-generation speculative-decoding harness, had been silently rejecting nearly all tokens from NVFP4-quantized draft models because Q, K, V and output projection scales were not being threaded through to the graph operations that consumed them. The result was speculative acceptance dropping to near zero on Nvidia FP4 hardware paths. b10701 passes the missing scales end-to-end and returns DFlash2 to design behavior on NVFP4 drafts.
Intel, backend memory, and a breaking CLI rename
b10692 at 12:22 UTC splits long rows in the Intel oneAPI SYCL TOP_K operator across multiple work-groups rather than one-per-row, closing a pathological slowdown on rows with tens of thousands of elements. b10699 at 18:07 UTC introduces ggml_backend_op_alloc_size_may_expand(), a backend hook that lets Metal, SYCL and WebGPU report per-operation extra temporary memory requirements at runtime; the RPC backend now queries remote servers on demand rather than relying on hardcoded lists. b10700 at 18:33 UTC renames the long-form --tensor-read-lazy CLI flag to --lazy-mode and adds the -lzm shorthand, a breaking change for anyone scripting the old flag; b10705 at 23:15 UTC follows through with improved TENSOR_READ_LAZY handling by forcing lazy tensors onto the CPU when lazy mode is enabled.
The pattern
llama.cpp has been the default reference for open-source inference throughout 2026, and the release cadence has drifted from weekly to daily and then, on days like this Saturday, to hourly. The 18-tag sequence spans every actively supported silicon family, closes two silent correctness bugs, adds one new hardware backend feature, and delivers a measurable throughput lift on a live Chinese frontier-open model. It also does the unglamorous work of a rename and a lazy-tensor CPU fallback the same night. Reading the release feed for a day like this is the closest thing available to a live view of where consumer AI inference is actually being made faster.