The RAMGPT library / 33 articles
Diagnose common installation, performance, and quality problems.
Diagnose GGML_ASSERT(ret.axis != GGML_BACKEND_SPLIT_AXIS_UNKNOWN) by separating tensor-split graph incompatibility from ordinary VRAM problems, then testing layer split, KV types, model paths and revisions.
If a Qwen3.8-Flash-Next MTP draft GGUF fails on output_hc_norm.weight, separate target-model loading from detached draft-head compatibility, inspect the sidecar tensor layout, and verify the exact llama.cpp MTP implementation.
Diagnose llama.cpp File Not Found errors by checking the actual GGUF path, process working directory, split-model shards, router configuration, and file readability before blaming model compatibility.
Current llama.cpp supports MTP speculative decoding through draft-mtp. Learn how to verify model support, enable it, isolate failed MTP contexts, and measure whether it actually improves your workload.
Fix the Qwen3.5-family blk.32.attn_norm.weight missing-tensor error when config metadata declares an MTP/NextN layer that the checkpoint does not contain.
Fix the llama.cpp error unknown model architecture 'mllama' by recognizing that current llama.cpp does not implement the Mllama vision architecture and choosing a supported path.
A Sep 30 vLLM report traces a large H100 decode regression to FA4 ignoring SplitKV num_splits on SM90; the one-line kernel fix is merged upstream but not yet pinned by vLLM.
If you searched whether llama.cpp supports NVFP4, the answer depends on the checkpoint, converter, model architecture, backend, and quant-scale path. Use this checklist to identify the real blocker.
Find the largest llama.cpp context that fits your GPU by separating weights, KV cache, compute buffers, concurrency, auto-fit, and backend limits.
A Sep 29 vLLM bug shows that adapting OpenAI-style tool call IDs to Mistral by keeping only the last nine characters can reject valid requests or create duplicate IDs.
Fix the llama.cpp error 'LLAMA_SPLIT_MODE_TENSOR not implemented for architecture qwen4exp' by using a supported split mode and understanding the architecture gate.
Explains what dbirks/Qwen3.8-27B-W4A16-AutoRound actually contains, why vLLM is its native target, and what current llama.cpp conversion does with pack-quantized weights.
Fix llama.cpp check_tensor_dims failures by separating missing tensors from wrong shapes, then checking model metadata, converter provenance, runtime support, and MTP assumptions.
Fix llama.cpp compute pp buffer allocation failures by shrinking micro-batches, preserving device-memory headroom, checking backend limits, fit warnings, and speculative decoding.
A Sep 27 vLLM fix traces Qwen3-Omni multimodal failures to an M-RoPE offset double-count that can crash at prompt end or silently shift positions.
Fix llama.cpp KV-cache allocation failures by reducing context or concurrency, checking cache types and device placement, and reading the backend allocator error.
Diagnose llama.cpp's unknown model architecture 'glm5next' error, verify the runtime you are actually using, and check whether upstream support has landed.
vLLM 0.30.0 can fail before startup on Pixtral-based models because its release code imports a symbol renamed by Transformers 5.17; main already contains the compatibility fix.
A Sep 25 llama.cpp report finds MTP throughput on an RX 7800 XT dropping from 55.79 to 25.80 t/s across a 146-commit range; Linux RADV does not reproduce it.
llama.cpp issue #29313 and PR #29370 trace an EAGLE-3 acceptance collapse to stale GGML allocation plans that ignore changing tensor input/output lifetimes.
SGLang issue #40901 shows that incremental streaming can resend the full accumulated text on graceful abort; PR #40902 fixes the text and token delta bookkeeping.
A fresh vLLM fix traces a DeepSeek-V4 CUDA-graph startup assertion to one stale max_model_len copy left behind after KV-cache auto-fitting reduces context.
A Sep 21 vLLM bug report shows speculative decoding can overrun a shorter draft model's RoPE cache, causing CUDA illegal-memory-access errors and full engine restarts.
A Sep 20 llama.cpp report bisects an MTP speculative-decoding regression to CUDA MoE weighted-reduction fusion: draft acceptance drops from 0.82 to 0.48 and MTP becomes slower than no draft.
A Sep 19 llama.cpp fix traces OpenVINO large-context failures to the backend reporting SIZE_MAX instead of the GPU's single-allocation limit, causing GGML to request one oversized KV-cache buffer.
A Sep 18 vLLM fix traces a silent Mistral-Large-3 accuracy regression to config remapping: YaRN frequency interpolation was preserved, but its no-mscale flag was not reaching DeepSeek-style attention.
A Sep 15 llama.cpp Vulkan fix targets silent wrong results when mul_mat reads a slice of a larger cache, a failure more dangerous than a clean crash.
A source-backed llama.cpp troubleshooting map for unknown architectures, wrong tensor shapes, corrupted GGUFs, CUDA OOM, multimodal mistakes, and runtime regressions.
A fresh llama.cpp draft fix explains an off-by-one scheduler-capacity crash triggered by larger speculative-decoding graphs and --spec-draft-n-max.
A fresh llama.cpp Metal PR isolates a large-tensor addressing failure at the 2 GiB slice offset and replaces Tensor API slicing with explicit 64-bit address calculation.
A source-level look at the new vLLM GGUF plugin patch for DeepSeek-V4, the exact architecture-resolution failure it fixes, and why new model names can break before their tensors do.
Learn how to reduce local LLM VRAM usage with quantization, shorter context, KV-cache compression, CPU offloading, smaller models, and runtime tuning.
Learn why a local LLM runs slowly and how model size, GPU offloading, context length, memory bandwidth, CPU threads, and runtime settings affect speed.