vLLM Prefix Cache Salts: Who Can Reuse Prompt State?

Automatic prefix caching can reduce repeated prefill work in an LLM server. In a shared service, though, the central question is not only how quickly a cache hits. It is which requests should be allowed to share that hit.

The upstream vLLM design provides an optional cache_salt to control prefix-cache reuse. A useful evaluation checks reuse latency and the isolation configuration separately.

What vLLM puts in a prefix-cache key

According to the vLLM design document, the key for a completed KV-cache block includes its tokens, the hash of the preceding block, and additional identity material where appropriate. That additional material can include a LoRA identifier, a multimodal input hash, or a cache salt.

Two implications matter. First, a text fragment is not reusable merely because the words look similar: the tokenized prefix must match, including the earlier chat template and system prompt. Second, the design caches complete blocks, so a short partial suffix is not necessarily a cache hit.

Request A: [same system prompt][same large document][question A]
Request B: [same system prompt][same large document][question B]
           <------- potentially reusable full blocks ----->

Request C: [different system prompt][same document][question C]
           <--- early mismatch alters later block hashes --->

This is an illustration, not a promise about block placement in a particular model or version.

Why the salt is a security control

Imagine that different customers send private text to the same hosted engine. With unrestricted shared caching, request latency may reveal whether some guessed prefix was previously processed. vLLM’s security guidance documents a prefix-cache timing side channel and describes cache salting as a mitigation.

A cache_salt is incorporated into the first block’s hash. Identical prompts with different salts do not share the same prefix-cache lineage. Matching prompts with the same salt remain eligible for reuse, subject to other conditions and eviction.

Salting limits prefix-cache reuse; it does not provide process-level isolation between tenants.

The salt is not encryption. It does not replace authentication, permission checks, request isolation elsewhere, or sensible log retention.

An OpenAI-compatible Python client can pass this vLLM extension in the request body:

response = client.chat.completions.create(
    model="your-served-model",
    messages=[{"role": "user", "content": "Summarize these public release notes."}],
    extra_body={"cache_salt": "BASE64_OF_32_RANDOM_BYTES_FOR_TENANT"},
)

The string is a placeholder, not a usable secret. vLLM’s current security guidance recommends a cryptographically unpredictable, tenant-scoped secret (for example 256 random bits). An authenticated gateway should supply the right secret for each tenant rather than accepting arbitrary client-selected identities. Prefix salting does not replace authorization or full process isolation.

Global versus tenant-scoped reuse

A single global salt may maximize the opportunity for cross-request reuse, but also allows unrelated users to participate in the same cache timing domain. A tenant-scoped salt narrows reuse to authenticated members of a tenant. A task-scoped salt can narrow it further but sacrifices more opportunities to share prefill.

No design choice comes with a universal performance percentage. The observed outcome depends on workload similarity, cache size, block boundaries, prompt formatting, concurrency, and eviction. Benchmark the workload you actually intend to serve rather than extrapolating from an identical-prompt microbenchmark.

Hash choice is a second, different boundary

The current vLLM cache configuration documents sha256 as the default prefix-cache hashing algorithm and sha256_cbor for reproducible serialization across environments. It also lists non-cryptographic xxhash variants and warns that weaker collision resistance brings risks in multi-tenant systems.

A hash algorithm controls how cache identities are represented and protected against collisions. A salt controls which trust group should produce the same identity for the same prefix. Neither decision substitutes for the other.

Always check the options supported by the installed vLLM version before copying a CLI flag from older documentation.

A five-case diagnostic matrix

Use synthetic inputs only. Keep the model, prompt token count, server version, sampling parameters, and load condition as stable as possible:

Record client-visible latency, queueing, prefill, and available cache telemetry. Repeat runs; one faster time-to-first-token observation is not sufficient evidence of isolation or a vulnerability. The matrix is a suggested experiment, not a test run conducted for this article.

If a service reports a poor hit rate, check whether prefix tokens really match, whether salts match by design, whether caching is enabled, and whether memory pressure caused eviction. Do not remove tenant isolation just to increase a dashboard percentage.

Decision for operators

A single-user local server and a public multi-tenant endpoint may choose different cache-sharing boundaries even with identical vLLM software. Document the boundary, test it using fake data, and treat latency measurement as separate from access-control verification.

For request-level timing definitions, see queue, prefill, and decode metrics.

Primary sources

• vLLM: Automatic Prefix Caching
• vLLM: Security
• vLLM: Cache Configuration