Error index · 49 fixes
Fix an error.
Start with the exact message in your terminal.
Each fix opens with the quickest safe answer, then explains the cause and how to confirm it. Use your browser's find (Ctrl+F or ⌘F) to search this page for the error text you see.
llama.cpp
llama.cpp topic hub ↗- llama.cpp
failed to allocate buffer for rs cache: Fix Recurrent-State OOMFix llama.cpp startup failures reporting failed to allocate buffer for rs cache by separating recurrent-state memory from KV cache, GPU weights, and server concurrency.
- llama.cpp ARM BF16 Slow: Check the Scalar Fallback
If BF16 inference or multimodal encoding is dramatically slower than F16 or Q8_0 on ARM, current llama.cpp may be falling back to a scalar ggml_vec_dot_bf16 path when the CPU lacks a matching vector implementation.
- llama.cpp SPLIT_AXIS_UNKNOWN: Fix Tensor-Split Startup Crashes
Fix llama.cpp SPLIT_AXIS_UNKNOWN assertions by testing layer split, updating old builds, isolating tensor-distribution graph failures, and separating them from VRAM errors.
- Qwen3.8 MTP
output_hc_norm.weight not found: Current llama.cpp Fix StatusIf a Qwen3.8-Flash-Next MTP draft GGUF fails on output_hc_norm.weight, separate target-model loading from detached draft-head compatibility, inspect the sidecar tensor layout, and verify the exact llama.cpp MTP implementation.
- llama.cpp File Not Found: Fix GGUF Paths, Shards, and Router 404s
Diagnose llama.cpp File Not Found errors by checking the actual GGUF path, process working directory, split-model shards, router configuration, and file readability before blaming model compatibility.
- llama.cpp
blk.32.attn_norm.weightNot Found: Fix MTP MetadataFix the Qwen3.5-family blk.32.attn_norm.weight missing-tensor error when config metadata declares an MTP/NextN layer that the checkpoint does not contain.
- llama.cpp
unknown model architecture: mllama: What Actually Fixes ItFix llama.cpp's unknown model architecture mllama error by confirming the GGUF architecture, ruling out an old binary, and switching to a runtime that supports Mllama vision.
- llama.cpp qwen4exp
SPLIT_MODE_TENSOR not implemented: Update the RuntimeFix the qwen4exp SPLIT_MODE_TENSOR not implemented error by updating stale llama.cpp builds, verifying the actual binary, and using layer split as a fallback.
- llama.cpp check_tensor_dims: Fix
tensor not foundandwrong shapeFix llama.cpp check_tensor_dims errors by separating missing tensors from wrong shapes, testing the unquantized GGUF, and matching converter, config, and runtime revisions.
- llama.cpp
failed to allocate compute pp buffers: Fix -ub and VRAMFix llama.cpp compute pp buffer failures by lowering -ub/--ubatch-size first, then checking VRAM headroom, backend allocation limits, --fit, and speculation.
- llama.cpp
failed to allocate buffer for kv cache: Fix VRAM and ContextFix llama.cpp KV-cache allocation failures by lowering context or parallelism, leaving VRAM headroom, checking cache types, and reading the real allocator error.
- llama.cpp
unknown model architecture: glm5next: Update Your RuntimeFix llama.cpp's unknown model architecture glm5next error by running a build that includes merged GLM-5.3-Flash support and verifying the actual binary or bundled backend.
- llama.cpp RX 7800 XT Vulkan MTP 54% Slower: Pin and Bisect
If RX 7800 XT Vulkan MTP throughput drops after an update, pin the last known-good build and bisect the 146-commit regression window; issue #29410 remains open.
- llama.cpp EAGLE-3 Acceptance Collapse: Backend Sampling Workaround
If EAGLE-3 acceptance collapses only with backend sampling, A/B --no-spec-draft-backend-sampling; allocator fix PR #29370 remains open.
- llama.cpp MTP Acceptance Drop After CUDA MoE Fusion: Verify the Prompt First
Issue #29168 initially bisected an MTP acceptance drop to CUDA MoE fusion, but the reporter later found it was specific to one raw prompt and closed the issue.
- llama.cpp OpenVINO 4 GiB KV Allocation Limit: Update Past #29852
Fix OpenVINO large-context allocation failures by using llama.cpp master with #29852, which reports the GPU's real single-allocation limit so GGML can split buffers.
- llama.cpp Vulkan Wrong Results from Sliced KV Cache: Update #28956
Fix llama.cpp Vulkan wrong-result regressions from sliced cache views by updating past merged PR #28956, then compare the same GGUF and prompt against CPU.
- llama.cpp Errors and Fixes: Diagnose the Failure Before Tuning
Match common llama.cpp errors to the right fix: update stale runtimes, repair GGUF paths, isolate tensor mismatches, reduce KV or compute memory, and identify regressions.
- llama.cpp Speculative Decode Crash at Higher n_max: Scheduler Hash Fix
Diagnose llama.cpp speculative-decoding assertions when --spec-draft-n-max grows the graph past scheduler hash capacity; lower n_max as a workaround while PR #28972 remains open.
- llama.cpp Metal 2 GiB Slice Bug: Update to the Merged Fix
Fix llama.cpp Metal corruption around the 2 GiB slice-offset boundary by updating to a build containing merged PR #28748, then retesting the same large workload.
- llama.cpp CUDA vs Vulkan: Which Backend Should You Use?
A practical llama.cpp CUDA-vs-Vulkan guide: verify the backend actually loaded, compare device support and kernels, benchmark the same workload, and avoid mistaking CPU fallback for a backend result.
- llama.cpp
invalid argument: --no-mmap: Use --load-mode CorrectlyIf llama.cpp rejects --no-mmap, verify the binary and wrapper first. Current upstream deprecated the old flag in favor of --load-mode none, while wrappers and older builds may parse it differently.
- Does llama.cpp Support NVFP4? Check the Exact Path Before You Convert
If you searched whether llama.cpp supports NVFP4, the answer depends on the checkpoint, converter, model architecture, backend, and quant-scale path. Use this checklist to identify the real blocker.
- llama.cpp Max Context vs GPU Memory: Find the Real Limit
Find the largest llama.cpp context that fits your GPU by separating weights, KV cache, compute buffers, concurrency, auto-fit, and backend limits.
- llama.cpp MTP Support: Use draft-mtp and Diagnose Failures
Current llama.cpp supports MTP speculative decoding through draft-mtp. Learn how to verify model support, enable it, isolate failed MTP contexts, and measure whether it actually improves your workload.
vLLM & SGLang
vLLM & SGLang topic hub ↗- vLLM guided_json Removed: Fix Structured Outputs Requests
Fix vLLM structured-output requests after the guided_json and guided_decoding_backend API removal, and distinguish obsolete fields from XGrammar JSON Schema compilation errors.
- vLLM KV Cache Too Small: Fix max_model_len Startup Failures
Fix vLLM startup errors caused by insufficient KV-cache capacity. Separate weight OOM from long-context sizing, batch concurrency and CUDA graph memory.
- vLLM Tool Calling 400 or Empty tool_calls: Fix Parser and Template
Diagnose vLLM tool-calling failures by separating HTTP 400 startup-flag errors from tool_choice behavior, parser mismatches, chat templates, and malformed arguments.
- vLLM EmbeddingGemma 2 dimensions=768 Error: Omit dimensions for Full Width
If vLLM rejects EmbeddingGemma 2 dimensions=768 but returns 768 values by default, omit dimensions for full-width output and verify the vector length.
- vLLM 404
Model Does Not Exist: Fix --served-model-nameFix vLLM 404 model-not-found errors by reading /v1/models, using the exact data[].id, and correcting --served-model-name or the client endpoint.
- vLLM LLM.generate() Hangs with msgspec Error: Fix SamplingParams Types
Fix a vLLM 0.31.0 LLM.generate() hang when msgspec reports Expected int, got float by validating integer SamplingParams fields before generation.
- vLLM DeepGEMM
Runtime Is Already Registered: Update Past the FixFix vLLM DeepGEMM's Runtime is already registered ImportError by using a build containing merged PR #48962, which moved vendored DeepGEMM away from the duplicate pybind11 registration path.
- vLLM H100 FA4 Decode Up to 48% Slower: Check the SplitKV Pin
Diagnose H100/SM90 FA4 decode regressions by checking the pinned FlashAttention commit: v0.31.0 still uses the pre-fix pin, while current vLLM main includes PR #198.
- vLLM Mistral
Duplicate tool call id: Stop Suffix TruncationFix Mistral tokenizer 400s and Duplicate tool call id failures by avoiding last-nine-character truncation; PR #59213 proposes stable unique 9-character mappings.
- vLLM Qwen3-Omni Position IDs Shift or Crash: Update the M-RoPE Fix
Fix Qwen3-Omni position-id length crashes and silent M-RoPE shifts with merged PR #58890; verify the exact build because release branches may not contain the merge.
- vLLM 0.30.0 Pixtral ImportError with Transformers 5.17: Fix It
Fix vLLM 0.30.0 PixtralRotaryEmbedding ImportError with Transformers 5.17. vLLM 0.31.0 contains the compatibility path for the renamed axial RoPE API.
- SGLang Duplicate Text on Streaming Abort: Disable Incremental Mode or Patch
Fix SGLang duplicate text after graceful abort: current main still replays full text in incremental mode, while proposed PR #40902 was closed unmerged.
- vLLM DeepSeek-V4 max_model_len Auto-Fit Crash: Update or Set It Explicitly
Fix DeepSeek-V4 CUDA-graph startup assertions after max_model_len auto-fit by using vLLM 0.31.0+ with the ModelState fix or setting an explicit context that fits.
- vLLM CUDA Illegal Memory Access: Check Draft vs Target Context
Diagnose vLLM speculative-decoding CUDA illegal-memory-access crashes by comparing target and draft context limits; route over-limit requests without speculation or align the limits.
- vLLM Mistral-Large-3 Accuracy Regression: Update the YaRN Fix
Fix the Mistral-Large-3 YaRN accuracy regression by using vLLM 0.31.0+ or a build containing PR #57563, which restores the missing no-mscale attention flag.
- vLLM GGUF
Unknown gguf model_type: deepseek_v4: Fix StatusDiagnose vLLM GGUF's Unknown gguf model_type: deepseek_v4 error, check whether plugin PR #126 is present, and separate architecture mapping from tensor-loading failures.
- Qwen3.8-27B W4A16 AutoRound: vLLM Artifact or GGUF?
Explains what dbirks/Qwen3.8-27B-W4A16-AutoRound actually contains, why vLLM is its native target, and what current llama.cpp conversion does with pack-quantized weights.
GPUs, CUDA & VRAM
GPUs, CUDA & VRAM topic hub ↗- PyTorch CUDA OOM: Allocated vs Reserved GPU Memory
Diagnose CUDA out-of-memory errors by separating live tensors, PyTorch's caching allocator, non-PyTorch allocations and fragmentation before changing batch sizes or allocator settings.
- torch.cuda.is_available() Is False: Fix PyTorch CUDA Detection
Fix torch.cuda.is_available() returning False by checking nvidia-smi, the active PyTorch wheel, Python environment, GPU visibility, and Docker passthrough.
- How to Reduce LLM VRAM Usage Without Breaking Performance
Learn how to reduce local LLM VRAM usage with quantization, shorter context, KV-cache compression, CPU offloading, smaller models, and runtime tuning.
- Local LLM Slow? Diagnose Prefill, Decode, GPU Offload, and Memory
Find why a local LLM is slow by separating prompt processing from token generation, checking GPU offload, memory pressure, context growth, and CPU bottlenecks.
Other tools
- Safetensors HeaderTooLarge: Diagnose the File Before Re-Downloading
A practical diagnosis of safetensors HeaderTooLarge errors: distinguish wrong file contents, incomplete transfers, and genuinely oversized metadata before changing your environment.
- Ollama Docker Connection Refused on Port 11434: Fix It
Fix Ollama connection refused errors from Docker by checking localhost, host.docker.internal, Linux host-gateway, OLLAMA_HOST binding, and Compose service DNS.
- Hugging Face 401/403 on Gated Models: Fix Access and Token Mismatch
Fix Hugging Face 401, 403, and GatedRepoError failures by checking repo approval, hf auth whoami, HF_TOKEN precedence, and the exact model path.