llama.cpp RX 7800 XT Vulkan MTP 54% Slower: Pin and Bisect
If RX 7800 XT Vulkan MTP throughput drops after an update, pin the last known-good build and bisect the 146-commit regression window; issue #29410 remains open.
Approximately 10 min read
Quick action: if Windows Vulkan MTP throughput suddenly halves on an RX 7800 XT, pin the last measured good revision first, then bisect on the same GPU, Windows driver, model, context depth, and MTP settings. Do not attribute the regression to PR #27952 just because it falls inside the range.
Issue #29410 remains open as of October 17, 2026. The report spans 146 commits and a Linux RADV tester did not reproduce the slowdown, so the root cause is not isolated yet.
In llama.cpp issue #29410, MTP throughput on a Windows RX 7800 XT fell from 55.79 to 25.80 tok/s at a 48K-token depth after a runtime update. Prefill barely changed, ordinary generation slowed, and draft acceptance remained 1.00.
The newer build included Vulkan int8 cooperative-matrix work from PR #27952, but the two revisions were 146 commits apart. The report does not isolate that patch as the cause. A second tester using an RX 7900 XTX with Linux RADV did not reproduce the slowdown.
The investigation therefore needs both a narrower revision range and a matched hardware/driver comparison. The measurements below come from the upstream report.
The exact reported setup
Issue #29410 compares these revisions:
| Tested build | Revision | Date |
|---|---|---|
| Older | c77ae695c |
September 17, 2026 era |
| Newer | 84e76d8a2 |
September 24, 2026 era |
The reporter explicitly notes that the newer revision is 146 commits after the older one. That matters because it prevents a causal shortcut: every performance change in that range remains a candidate until narrower testing eliminates it.
The machine is:
OS: Windows 11
GPU: AMD Radeon RX 7800 XT 16 GB
architecture: RDNA3
Vulkan driver: AMD proprietary
CPU: Ryzen 5 7600X
RAM: 32 GB
reported shared memory: 32768 bytes
matrix cores: KHR_coopmat
The model is described in the issue as a Qwen3.5/3.6 35B-A3B MoE model in Q4_K_M, with native MTP available.
For the empty-context benchmark, the reporter uses a llama-bench command equivalent to:
device: Vulkan0
GPU layers: 99
CPU MoE layers: 18
Flash Attention: on
prompt: 512 tokens
generation: 32 tokens
For the long-context MTP measurements, the server is configured with:
context: 163840
KV cache: q4_0 K + q4_0 V
MTP draft depth: 3
reasoning: off
batch / ubatch: 2048 / 2048
The long-context measurements are taken after filling the KV cache to approximately 48k and 130k tokens.
That is important context. This is not a tiny synthetic matrix-kernel result. It is a serving-style path with MoE, quantized KV, MTP, and substantial context depth.
What the reporter measured
These are the upstream issue author’s numbers. RAMGPT did not run this benchmark.
| Test | c77ae695c | 84e76d8a2 |
|---|---|---|
| pp512 | 1174 t/s | 1151 ± 9 t/s |
| tg32, no MTP | 42.4 t/s | 36.1 ± 0.6 t/s |
| MTP n=3, tg128 at 48k | 55.79 t/s | 25.80 t/s |
| MTP n=3, tg128 at 130k | 39.15 t/s | 27.14 t/s |
The 48k MTP point is the headline-sized change:
55.79 to 25.80 tok/s.
That is a reduction of about 54%.
But the shape of the whole table is more informative than the largest percentage.
Prompt processing is roughly flat:
1,174 to 1,151 tok/s.
Ordinary decode is slower:
42.4 to 36.1 tok/s.
MTP decode is much slower at both measured depths.
That means this is not simply “the new fast prefill kernel failed to improve pp512.”
There is a decode-side regression to explain too.
Acceptance stayed at 1.00
This detail sharply separates #29410 from the CUDA MTP exactness regression RAMGPT covered on September 20.
In that earlier case, draft acceptance collapsed and greedy output diverged. A speculative path that proposes bad candidates naturally loses performance because the target rejects more of the draft.
Issue #29410 says something different:
MTP acceptance remained 1.00 in both builds.
The draft tokens are still being accepted.
So the giant drop at 48k is not explained by “MTP suddenly became a bad drafter.”
Conceptually, the old run looks like:
The drafter proposes useful tokens, the target verifies them, and accepted tokens can provide a speedup.
The new run still has:
The proposed tokens still pass target verification.
but each speculative step is taking much longer.
That moves the investigation away from model quality and toward execution cost:
Kernel selection, tile shape, driver behavior, MoE dispatch, memory traffic, long-context attention cost, and work scheduling.
This distinction is exactly why acceptance rate belongs beside tokens per second in speculative-decoding diagnostics.
PR #27952 is relevant, but it is not yet convicted
The newer revision contains merged PR #27952:
vulkan: int8 coopmat1 matmul implementation for AMD RDNA3 and RDNA4
The design makes sense.
Before the change, llama.cpp’s Vulkan cooperative-matrix path mainly relied on fp16 matrix operations after dequantizing weights into shared memory.
Many AMD GPUs have strong int8 matrix throughput, so the new path performs quantized MMQ work with int8 cooperative-matrix instructions.
The PR supports several common quant formats, including:
Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, Q3_K, Q4_K, Q5_K, Q6_K, MXFP4, NVFP4, and IQ4_NL.
That makes it directly relevant to the Q4_K_M model in #29410.
The upstream PR benchmarks also show why this work was attractive.
On a Radeon 8060S Strix Halo system, the PR author reported Qwen3.6-35B-A3B Q4_K_M pp512 moving from:
777.8 to 1,023.6 tok/s.
about a 1.32x improvement.
On an RDNA4 Radeon AI PRO R9700, the same model/quant row was reported as:
2,760 to 3,365 tok/s.
about a 1.22x improvement.
Those are PR-author measurements, not RAMGPT measurements, and they are not the same hardware or driver as issue #29410.
That difference is the entire point.
The Windows result does not match the Strix Halo story
The RX 7800 XT report expected some benefit because both systems sit in the broader RDNA3 family and the new path is explicitly enabled there.
Instead:
| Configuration | Reported effect |
|---|---|
| Strix Halo, Q4_K_M MoE, pp512 | Strong gain |
| RX 7800 XT on Windows, Q4_K_M MoE | Nearly flat prefill, slower decode, much slower MTP |
This is a classic case where “same architecture family” is not enough information.
The actual execution environment also includes:
Integrated versus discrete hardware, shared-memory limits, drivers, tile choices, MoE shapes, context depth, quantization layout, and dispatch thresholds.
A shader that is well-tuned for one RDNA3 configuration can choose a less favorable tile or occupancy point on another.
That does not prove that this is what happened here. It tells us what kind of hypothesis is worth testing.
Linux RADV provides an important negative control
A second commenter tested another RDNA3-family card:
The Linux control used an RX 7900 XTX, Mesa RADV, and dense Qwen3.8-27B IQ3_S.
Their interleaved llama-bench runs compared a pre-#27952 build with a current master containing the new path.
They reported:
| Test | Before #27952-era merge | Master containing #27952 |
|---|---|---|
| pp512 | 943.9 t/s | 980.2 t/s |
| tg128 | 51.11 t/s | 50.98 t/s |
For their MTP probe, median generation throughput was reported as:
79.6 to 82.0 tok/s.
with overlapping acceptance ranges.
In other words: no comparable decode or MTP regression.
That does not exonerate every commit in the 146-commit range, because the cross-check also changes the GPU, operating system, driver, model type, quant, and workload.
But it is useful evidence against a universal statement like:
“RDNA3 Vulkan is now slower.”
The failure surface is narrower.
The 32 KB shared-memory clue
The most specific upstream hypothesis so far comes from the contributor behind #27952.
Responding to the Windows report, they noted that the optimization was tuned and tested on Linux, and called out the Windows device’s reported shared-memory limit:
The Windows RX 7800 XT reported 32,768 bytes of shared memory.
The Linux RADV cross-check reports:
65,536 bytes of shared memory.
The contributor suggested that the 32 KB limit may restrict the Windows path to a medium tile and that it might need a smaller L-warptile.
That is a useful source-level hypothesis because tile size, cooperative-matrix packing, and shared-memory budget are directly connected.
But it is still a hypothesis.
There is no upstream A/B in #29410 yet showing:
A confirming test would keep the RX 7800 XT, Windows driver, and build fixed, change only tile selection, and check whether performance recovers.
Until that test exists, the responsible wording is:
The shared-memory/tile explanation is plausible and comes from the patch author, but it is not yet a confirmed root cause.
Why the 146-commit range matters so much
The issue author is careful about this, and we should be too.
Between the old and new revisions, several Vulkan changes landed. The report itself mentions work involving IQ formats, symbol visibility, Intel Xe Flash Attention, Adreno, convolution alignment, and other MMQ changes.
Some appear unlikely to be hot for this exact Q4_K_M workload.
“Unlikely” is not a bisect.
A clean performance investigation should reduce the range rather than reason from commit titles alone.
At minimum, I would want these four builds:
- Last known-good revision.
- Revision immediately before #27952 merged.
- The #27952 merge commit.
- The current revision being investigated.
Then run the exact same command and warmup procedure.
That separates:
That bounds whether the regression precedes, coincides with, or follows #27952, including interactions with surrounding changes.
Without that split, reverting one attractive suspect and seeing a speed change can still mislead if surrounding changes interact.
A better A/B matrix for AMD Vulkan
If I had access to the affected machine, I would keep the test small and systematic.
1. Separate prefill from decode
Run llama-bench with:
Measure pp512 separately from tg32 or tg128.
Do not summarize them into one “overall speed” number.
The issue already shows that prefill and decode can move in different directions.
2. Test MTP with acceptance beside throughput
Record:
Record effective generation throughput, draft acceptance, mean accepted length, and context depth.
If acceptance remains at 1.00 while throughput changes, the problem is execution cost rather than draft quality.
3. Hold context depth fixed
The issue reports both 48k and 130k.
That is useful because long context changes attention and KV behavior. A regression that appears only at one depth may point somewhere very different from a regression that is flat across all depths.
4. Compare dense and MoE models
The Linux negative control used a dense model while the Windows failure uses a 35B-A3B MoE model.
A same-card dense-vs-MoE comparison would help answer whether the slow path is tied to expert matmuls or is more general.
5. Compare the exact #27952 boundary
This is the most valuable missing test.
Everything else is still inference until the commit range gets smaller.
Do not confuse this with the earlier Vulkan prefill fast-path bug
RAMGPT has already covered a Vulkan case where a layout guard silently disabled a fast Flash Attention path for hybrid KV-cache views.
That September 2 article was about:
The earlier case rejected a valid cache view at a layout guard, preventing fast-path selection and slowing prefill.
Issue #29410 reports a different configuration:
RDNA3 int8 cooperative-matrix execution on the Windows proprietary driver, using RX 7800 XT, MoE Q4_K_M, and MTP.
The differing hardware, driver, and failing inference phase require separate tests.
“Vulkan performance regression” is too broad a category to be useful.
The hardware, driver, quant, model architecture, and phase of inference are part of the bug report.
Do not confuse it with the CUDA MTP exactness regression either
The September 20 CUDA MoE article had this signature:
The CUDA case combined falling acceptance, changed greedy output, and slower MTP.
The Windows Vulkan report has:
The Windows Vulkan case retained acceptance 1.00 while MTP slowed; the report did not identify an output-quality failure.
Those are different failure classes.
The first points toward numerical/exactness behavior in verification.
The second points toward per-step runtime cost.
That one metric — acceptance — changes the investigation.
What operators should do today
If you run llama.cpp on AMD Vulkan and a recent update makes MTP or decode slower, record the exact revision before changing model settings.
Then capture:
Record GPU, OS, Vulkan driver, reported shared memory, architecture, quantization, context, CPU-MoE placement, MTP depth, acceptance, and separate prompt-processing and generation rates.
If the regression resembles #29410, compare a known-good build against a build immediately around the #27952 merge instead of simply disabling MTP and concluding that speculative decoding is broken.
For production use, the conservative choice is the revision that is already measured as stable on your own workload.
That is not a recommendation to pin forever. It is a way to keep serving predictable while the upstream range is narrowed.