Troubleshooting

llama.cpp SPLIT_AXIS_UNKNOWN: Fix Tensor-Split Startup Crashes

Fix llama.cpp SPLIT_AXIS_UNKNOWN assertions by testing layer split, updating old builds, isolating tensor-distribution graph failures, and separating them from VRAM errors.

Approximately 3 min read

Quick fix: if --split-mode tensor aborts with:

GGML_ASSERT(ret.axis != GGML_BACKEND_SPLIT_AXIS_UNKNOWN) failed

retry the same model and device ratio with layer split. If layer split works, you have isolated the failure to the tensor-distribution path rather than ordinary VRAM capacity.

Also check your llama.cpp revision. The original Qwen3.8-Flash-Next report used build 10680; current llama.cpp now allows qwen4exp through the tensor-split architecture gate, and current upstream reports show the model running under -sm tensor. A stale binary and a current tensor-graph bug are different problems.

The assertion comes from graph split-state analysis: an operation’s input tensors reached a distribution combination the runtime could not reconcile.

Keep the model, GPUs, and device ratio fixed and change only the distribution strategy:

Run Options Record
Tensor split -sm tensor -ts 1,1 First failing operation and assertion
Layer split -sm layer -ts 1,1 Whether initialization and generation succeed

In issue #27964, Qwen3.8-Flash-Next build 10680 failed under tensor split while layer split with the same ratio served requests. The issue remains open, but the report itself says it had not been retested on newer master. Treat it as a versioned reproduction, not a permanent statement that qwen4exp cannot use tensor split.

How an operation gets an unknown split state

The meta backend tracks whether tensors are split along a dimension, mirrored, partial, or unresolved. Generic operation handlers examine their input states and try to derive a valid output state.

Architecture-specific graphs can expose combinations that the handler cannot reconcile. In issue #26902, the reporter traced the failure to a multiply whose attention-gate input used a different split granularity from the attention output.

The proposed local adjustment made those states agree for that model. Applying the same rule to an unrelated architecture would require checking its tensor shapes and distribution semantics.

The ratio 1,1 only controls the relative assignment to devices. It does not establish that every operation supports the selected distribution strategy.

Test cache format independently

Issue #27116 reproduced the assertion with tensor split and IQ4_NL keys and values on two RTX 5060 Ti GPUs.

If your command uses quantized KV, compare the existing configuration with an F16 cache while retaining the same split mode. Then compare layer split with the original cache types. Changing these independently helps distinguish a general tensor-split failure from a cache-format interaction.

Remove draft models, speculative decoding, and custom tensor overrides unless they are necessary to reproduce the crash. The report in #27964 was particularly useful because the failure occurred without DFlash2 or MTP.

Bound a regression with exact builds

Save the executable version:

llama-server --version

For a source checkout, also save:

git rev-parse HEAD

Issue #25829 describes a regression across a build boundary. If a known-good revision is available, run the same model and command on both revisions.

An upstream report should include the GPU models, backend, exact GGUF, command, split ratio, KV types, first failing operation, and layer-split result. That gives a maintainer a concrete graph-placement comparison to reproduce.

An allocator error naming a failed memory buffer needs a different investigation. For this assertion, repeated context reductions or model downloads provide little evidence unless they change the failing graph in a reproducible way.

Sources and further reading