DSpark PCTree on an RTX 4090: Higher Acceptance, Lower Throughput

Speculative decoding has an appealing promise: spend a little extra work on a cheap draft path, verify several tokens with the target model at once, and turn expensive sequential decoding into something more parallel.

DSpark pushes that idea further. Instead of running a conventional autoregressive draft model token by token, it predicts a block in one draft pass and uses a lightweight Markov head to recover some left-to-right dependency inside the block.

The new Parent-Conditioned Drafting Tree, or PCTree, asks an obvious next question: if DSpark’s Markov head can score continuations conditioned on different parents, why follow only one linear draft chain?

The PCTree paper reports gains over matched DSpark ranging from 3.1% to 29.5% across tested Qwen3 configurations and benchmarks. A fresh experimental llama.cpp fork implements the idea for CPU and CUDA.

I wanted to know what happens on a consumer RTX 4090.

The answer was not what the headline numbers might suggest.

On this setup, ordinary linear DSpark was excellent. It more than doubled Qwen3-8B Q8 decode throughput. PCTree then raised speculative acceptance substantially, but every tree configuration tested remained slower than linear DSpark overall.

That does not make PCTree a failed idea. The logs show something more interesting: tree management itself is cheap, the GPU amortizes much of the wider verifier workload, and the useful metric is not acceptance alone. The real question is whether the extra accepted path length is large enough to pay for each more expensive verification round.

This article documents the test, the parameter sweep, the internal timing data, and a public reproduction script.

What PCTree changes in DSpark

Standard speculative decoding usually works with a draft sequence and a target verifier:

Draft:   A → B → C → D
Target:  A ✓ B ✓ C ✗

Once the target disagrees at C, the remaining suffix is no longer useful.

DSpark makes drafting cheaper by predicting a block semi-autoregressively, but the final speculative candidate is still normally a single chain.

PCTree uses the already-trained DSpark Markov head to preserve several parent-conditioned continuations:

              A
           /  |  \
          B   X   Y
         / \      \
        C   D      Z

The target model can then verify a prefix-closed tree rather than one chain. If the highest-probability chain diverges early, another branch may still match the target.

The fork exposes two useful controls:

• k: local branching/frontier width
• N: verifier-node budget, including the root

A larger tree can improve the chance that the target path exists somewhere in the draft, but every extra verifier node also has a cost.

That trade-off is the entire story of this benchmark.

Test setup

The tested runtime was the experimental getraid/llama.cpp-dspark-pctree branch, built for CUDA with the RTX 4090’s SM89 architecture.

The model pair came from the official ggml-org/Qwen3-8B-GGUF repository:

The principal multi-workload benchmark used 32 prompts across eight categories, with two measured runs per prompt for each configuration:

• coding
• math/reasoning
• summarization
• retrieval/context QA
• technical writing
• extraction
• multilingual
• longer-context analysis

That produced 64 measured requests per configuration and 256 measured requests total across Plain, Linear DSpark, PCTree k3/n16, and PCTree k4/n22.

The main performance number is weighted decode throughput:

all generated output tokens
---------------------------
all measured generation time

Model loading and prompt evaluation are excluded from this figure. This is therefore decode/output-token throughput, not full request latency.

Result 1: linear DSpark was a major win

The first result is easy to miss because PCTree is the new feature: DSpark itself worked extremely well on this 4090/Q8 setup.

Linear DSpark delivered 2.106× the plain model’s decode throughput, an increase of about 110.6%.

For a local inference user, that is already the biggest practical finding in this experiment.

Result 2: PCTree acceptance improved, but throughput fell

PCTree did what it was supposed to do algorithmically.

Mean acceptance rose from 31.29% with linear DSpark to 45.08% with k3/n16 and 46.68% with k4/n22.

The k4/n22 configuration therefore raised mean acceptance by about 49% relative to linear DSpark.

Yet its weighted decode throughput was 19.9% lower than linear DSpark.

The k3/n16 configuration was 21.4% slower than linear DSpark.

This is the key lesson:

Higher speculative acceptance is not the same thing as higher speculative-decoding throughput.

The verifier still has to process the tree, and a wider verified batch is only worthwhile when the extra accepted progress saves enough future rounds.

Across the eight workload categories in the 32-prompt benchmark, neither tested PCTree configuration beat linear DSpark in aggregate throughput.

Why the bigger k4/n22 tree was slightly faster than k3/n16

At first glance, k4/n22 should look worse. It verifies roughly 22 nodes per round instead of 16.

The server logs show why it still gained a little throughput.

Averaged over the measured requests:

The important line is the verifier cost.

Increasing the tree from about 16 to about 22 nodes increased target time by only around 4.7%, not 37.5%.

That is evidence that the RTX 4090 is amortizing a large portion of the wider verification batch in parallel.

Meanwhile, mean accepted length increased a little more than the measured round cost. That was enough for k4/n22 to edge ahead of k3/n16, even though both remained well behind linear DSpark.

Also note what did not dominate the timing: tree selection itself.

Typical selection time was only around 0.4 ms per round. The bottleneck is not simply “building a tree on the CPU.” The dominant costs remain the draft and target computations needed for each speculative round.

Parameter sweep: the tested PCTree sweet spot was k5/n32

A second-stage sweep reduced the workload to one representative prompt from each of the eight categories and tested wider trees.

The curve finally turned over.

Throughput improved from k3/n16 through k5/n32, then declined at k5/n40 and k6/n48.

Among the tested configurations, k5/n32 was the PCTree sweet spot.

But it was still about 16.0% slower than linear DSpark on the same eight representative prompts.

Acceptance kept climbing after throughput peaked

The acceptance sweep makes the trade-off even clearer.

The highest acceptance configuration was k6/n48.

The highest PCTree throughput configuration was k5/n32.

Those are not the same configuration.

That is exactly why acceptance rate is useful diagnostic data but a poor standalone optimization target.

What a 32-node round cost on the RTX 4090

The k5/n32 log is particularly informative because it sits near the measured throughput peak.

Across the measured rounds, the internal PCTree costs were approximately:

draft       ~8.99 ms / round
target     ~13.87 ms / round
select      ~0.42 ms / round
update      ~0.42 ms / round
KV          ~0.15 ms / round
--------------------------------
total      ~23.85 ms / round

The run was verifying almost exactly 32 nodes per round with roughly 12.4 leaves per round.

Again, selection was a small fraction of the total cost.

The verifier’s ability to process more nodes sub-linearly is what allowed PCTree throughput to improve from n16 to n32. Beyond that point, the extra accepted progress no longer paid for the wider tree.

Why this differs from the PCTree paper and RTX 5090 results

The PCTree paper reports relative gains over matched DSpark ranging from 3.1% to 29.5% across its tested Qwen3 models and benchmarks.

The llama.cpp research issue also includes a positive RTX 5090 example using the official Qwen3-8B pair in BF16. In that single story-prompt test, linear DSpark averaged about 155.6 tok/s, while k3/n16 reached about 175.3 tok/s.

That does not conflict with this RTX 4090 result because the experiments are not apples-to-apples.

Important differences include:

• RTX 5090 versus RTX 4090
• SM120 versus SM89
• BF16 versus Q8 target/draft weights
• different prompts and workload mixtures
• potentially different kernel efficiency and verification-batch behavior

The research issue itself also reports cases where PCTree loses badly, including a larger Qwen model on the RTX 5090.

The useful conclusion is not that one benchmark is right and another is wrong.

It is that PCTree performance is highly sensitive to the cost curve of target verification on the exact model, precision, workload and GPU.

A tree is only profitable when additional verifier width is cheaper than the rounds it saves.

A note on exact output matching

The tests used temperature zero, but exact output hashes were not consistently repeatable even for the non-speculative Q8 CUDA baseline across the larger prompt set.

For that reason, this experiment did not use bit-identical output hashes as a PCTree-specific correctness criterion.

That matters because it would be easy to observe a token-stream difference between linear DSpark and PCTree and incorrectly conclude that the tree implementation caused it. Under this particular quantized GPU setup, exact reproducibility was already imperfect without PCTree.

For a stricter losslessness/correctness study, a separate high-precision target test with an explicitly controlled deterministic configuration would be a better methodology.

Reproduce the sweep

The benchmark harness used for the representative parameter sweep is available directly from RAMGPT.org:

• Download pctree_sweep.py
• Download the multi-workload summary CSV
• Download the parameter-sweep summary CSV

The script intentionally uses only the Python standard library. It launches llama-server sequentially for each configuration, performs a warm-up request, runs the eight representative workloads, records server logs, and writes raw and summarized CSV output.

The expected model pair is the official Qwen3-8B Q8 target plus DSpark Q8 draft from ggml-org/Qwen3-8B-GGUF.

A matching PCTree server can be built from the experimental fork with an Ada-specific CUDA build such as:

cmake -B build \
  -DGGML_CUDA=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CUDA_ARCHITECTURES=89

cmake --build build --target llama-server -j"$(nproc)"

Then run:

python3 pctree_sweep.py

The script does not expose or require any public inference endpoint. Everything runs against a local 127.0.0.1 llama-server instance.

What I would use on an RTX 4090 today

For this specific Qwen3-8B Q8 setup, the answer is straightforward:

Use linear DSpark.

It delivered roughly 216 output tokens per second, compared with roughly 102 tok/s without speculative decoding.

PCTree is technically interesting and clearly improves draft acceptance, but the best tree configuration tested here still gave away about 16% of the throughput that linear DSpark achieved.

That may change with different precision, future CUDA kernels, different models, longer DSpark blocks, or future revisions of the experimental fork.

The more durable lesson is broader than PCTree:

In speculative decoding, optimize accepted progress per unit of verification cost, not acceptance rate in isolation.

On this RTX 4090, widening the tree from 16 to 32 nodes improved the trade-off. Widening it further increased acceptance without increasing useful throughput.

That is exactly the kind of hardware-dependent behavior that makes real local inference benchmarks worth running.