llama.cpp v0.6.0 Extended Batch API: A Migration Checklist
llama.cpp v0.6.0 adds llama_batch_ext and llama_process for mixed token and embedding input. Here is what C API integrators need to test before upgrading.
Open-Source AI Commentary Contributor
MapleKernel
MapleKernel is a pseudonymous contributor covering the open-source AI ecosystem, with particular attention to inference runtimes, quantization, implementation changes, developer communities, and the technical consequences of competing design decisions. Articles combine source-level analysis with commentary on how open-source projects evolve in practice.
Published work
Articles
17 published
vLLM Hybrid Mamba Prefix Caching Can Cost Throughput Before a Cache Hit
vLLM issue #60008 reports an 11–16% output-throughput cost from Mamba align-mode prefix caching with zero cache hits, exposing host-side launch and prefill-splitting overhead.
By MapleKernel ·
llama.cpp Host-RAM MoE Pipeline Parallelism: What PR #29963 Changes
A source-level look at llama.cpp PR #29963, which explores pipeline parallelism when MoE expert weights remain in host RAM and must be uploaded to the GPU that owns each layer.
By MapleKernel ·
llama.cpp K-Pool Graph Reallocation: The GLM-5.3/Qwen4exp Fix
Merged llama.cpp PR 29958 stabilizes k-pool graph shapes for GLM-5.3 and Qwen4exp, addressing reallocation after mutable state changed the reserved graph.
By MapleKernel ·
llama.cpp SPLIT_AXIS_UNKNOWN: Fix Tensor-Split Startup Crashes
Fix llama.cpp SPLIT_AXIS_UNKNOWN assertions by testing layer split, updating old builds, isolating tensor-distribution graph failures, and separating them from VRAM errors.
By MapleKernel ·
Qwen3.8 MTP 'output_hc_norm.weight not found': Current llama.cpp Fix Status
If a Qwen3.8-Flash-Next MTP draft GGUF fails on output_hc_norm.weight, separate target-model loading from detached draft-head compatibility, inspect the sidecar tensor layout, and verify the exact llama.cpp MTP implementation.
By MapleKernel ·
llama.cpp File Not Found: Fix GGUF Paths, Shards, and Router 404s
Diagnose llama.cpp File Not Found errors by checking the actual GGUF path, process working directory, split-model shards, router configuration, and file readability before blaming model compatibility.
By MapleKernel ·
vLLM H100 FA4 Decode Up to 48% Slower: Check the SplitKV Pin
Diagnose H100/SM90 FA4 decode regressions by checking the pinned FlashAttention commit: v0.31.0 still uses the pre-fix pin, while current vLLM main includes PR #198.
By MapleKernel ·
Qwen3.8-27B W4A16 AutoRound: vLLM Artifact or GGUF?
Explains what dbirks/Qwen3.8-27B-W4A16-AutoRound actually contains, why vLLM is its native target, and what current llama.cpp conversion does with pack-quantized weights.
By MapleKernel ·
llama.cpp check_tensor_dims: Fix 'tensor not found' and 'wrong shape'
Fix llama.cpp check_tensor_dims errors by separating missing tensors from wrong shapes, testing the unquantized GGUF, and matching converter, config, and runtime revisions.
By MapleKernel ·
vLLM Qwen3-Omni Position IDs Shift or Crash: Update the M-RoPE Fix
Fix Qwen3-Omni position-id length crashes and silent M-RoPE shifts with merged PR #58890; verify the exact build because release branches may not contain the merge.
By MapleKernel ·
llama.cpp EAGLE-3 Acceptance Collapse: Backend Sampling Workaround
If EAGLE-3 acceptance collapses only with backend sampling, A/B --no-spec-draft-backend-sampling; allocator fix PR #29370 remains open.
By MapleKernel ·
Nemotron-3-Puzzle 75B-A9B Lands in llama.cpp — MTP Is Still Missing
llama.cpp PR #25444 adds Nemotron-3-Puzzle-75B-A9B GGUF support, including heterogeneous per-layer MoE metadata, converter fixes, and model loading — but not the model's MTP draft head yet.
By MapleKernel ·
Why a Small Vulkan Fusion Gave Gemma 4 About 4% in llama.cpp
A source-level look at llama.cpp PR #28024: how fusing RMSNorm, residual math, RoPE, views, and row writes cuts Vulkan graph overhead in Gemma 4.
By MapleKernel ·
Ollama and llama.cpp: It's Complicated
A source-driven history of the increasingly uncomfortable relationship between Ollama and llama.cpp: upstream engineering, attribution disputes, forks, compatibility failures, community backlash, and the cost of hiding the engine.
By MapleKernel ·
The One-Line Vulkan Fix Hiding an 83% Prefill Gain
A llama.cpp Vulkan guard rejected valid hybrid-model KV-cache views, silently disabling a faster Flash Attention path for quantized caches.
By MapleKernel ·
llama.cpp vs ik_llama.cpp: The Fork, Drama and Performance Race
A lively look at how ik_llama.cpp split from llama.cpp, why local-AI users care, what each project does best, and why the 'fork drama' is more useful than it sounds.
By MapleKernel ·