llama.cpp K-Pool Graph Reallocation: The GLM-5.3/Qwen4exp Fix

A GLM-5.3-Flash user reported a previously working 307,200-token context failing after llama.cpp commit c2ed48576. The reported RTX 5090 configuration used ubatch 8,192 and about 30 GiB before the change; afterward, context creation failed with:

allocating 22408.74 MiB on device 0: cudaMalloc failed
graph_reserve: failed to allocate compute buffers
failed to allocate compute pp buffers

The reporter could start at a reduced context of 204,800. These figures come from issue #29950.

PR #29958, merged October 5, 2026, addresses the graph-shape problem behind the k-pool reservation changes.

The reserve stopped matching later graphs

The PR identifies two decisions that depended on mutable state:

• Qwen4exp: sharing sequence cells changed the pooled-key path, removing twelve graph nodes and one leaf from a later graph.
• GLM5-Next: current token and KV counts selected between gather and dense paths, producing different graph shapes for decode and prompt processing.

A different graph shape triggered scheduler reservation again. That new reservation could lose the earlier worst-case sizing, allowing subsequent state growth to exceed the reserved capacity.

Commit c2ed48576 attempted to cover future growth by reserving worst-case k-pool geometry at initialization. The large-context issue illustrates the cost: prompt-processing workspace could become too large before the cache actually reached that state.

The merged change fixes shape selection

PR #29958 makes graph shape constant for a given context. Qwen4exp and GLM5-Next consistently create the pooled-key representation, scatter fresh keys, and gather the pools. GLM gather selection uses context constants instead of current n_tokens and n_kv.

The intended result is that the reserve and later decode graphs retain compatible topology, avoiding a decode-time reserve that silently changes the capacity assumptions.

The contributor supplied this regression command:

GGML_SCHED_DEBUG_REALLOC=1 ./bin/llama-batched-bench -hf ggml-org/GLM-5.3-Flash-GGUF:Q2_K -npp 2500 -ntg 32 -npl 1,2 -c 32768 -pps -kvu

Before the fix, the reported reproduction aborted on unexpected graph reallocation. The PR reports passing afterward, including variants with different prompt and ubatch sizes, a Qwen4exp equivalent, architecture tests, and recurrent-state tests.

Apple M2 Ultra measurements in the PR also favored the selected dense path over gather in the tested cases. Those timings describe the contributor’s tests; they do not establish how the original RTX 5090 capacity boundary changed.

Check a packaged build against the fix

If a GLM-5.3 or Qwen4exp update introduces compute-buffer OOM, determine whether the binary includes #29958. Compare the same GGUF, context, batch, ubatch, placement, and backend against a known-good revision.

Reducing ubatch can relieve prompt-processing workspace pressure, with a possible prefill-throughput cost. Keep that temporary setting in the comparison record rather than attributing its effect to a runtime fix.

Distinguish failed to allocate compute pp buffers from failed to allocate buffer for kv cache. The first concerns graph workspace; the second names cache allocation. The unchanged GGUF size does not rule out a runtime memory regression in either allocation.

For the original 307,200-token workload, restoration of the former fit boundary still requires a matching post-fix run. The merged graph tests establish the reservation behavior they exercise, not that separate capacity result.