llama.cpp Q4_K Prefill Fusion: Gate, Up, and SwiGLU in One CUDA Path
A new llama.cpp CUDA PR targets duplicate activation quantization and intermediate FP32 traffic in dense Q4_K MLP prefill, with contributor-reported gains from 4.1% to 14.7%.
The RAMGPT library / 3 articles
AI Hardware
Choose CPUs, GPUs, memory, and storage for AI workloads.
Latest in AI Hardware
All topics ↗Why llama.cpp's Blackwell NVFP4 Prefill Was Starving Tensor Cores
A new llama.cpp CUDA patch shows that Blackwell NVFP4 prefill was limited by tile delivery, barriers, and register pressure—not simply tensor-core arithmetic.
How Much VRAM Do You Need for Local LLMs?
Learn how to estimate LLM VRAM requirements from model size, quantization, KV cache, context length, runtime overhead, and GPU offloading.