vLLM EmbeddingGemma 2 dimensions=768 Error: Omit dimensions for Full Width

Quick answer: If google/embeddinggemma-2 in vLLM returns a 768-dimensional vector when no output dimension is specified, but rejects an explicit dimensions=768 request with a stated maximum of 512, you may be hitting the validation inconsistency recorded in vLLM issue #60977.

For affected versions, the least intrusive workaround for a full-width embedding is to omit the dimensions parameter, then verify that the returned vector really has 768 elements. Do not silently truncate or pad vectors to evade validation.

The error and the apparent contradiction

The reported error is:

VLLMValidationError: Model 'google/embeddinggemma-2' only supports
dimensions in range [1, 512], got 768.

The issue reporter recorded this outcome:

These observations came from an upstream development build around vLLM 0.30.1rc1, using the pooling runner and BF16 on an RTX A5000. They are not a blanket statement about all checkpoints or future releases.

An embedding dimension is the number of numeric components in the returned vector. For Matryoshka-style embeddings, some models support requesting a smaller leading subvector for storage or retrieval cost savings.

If a system accepts the native 768-vector by default, it is reasonable to expect an explicit request for that native width to behave consistently. The affected validation path did not.

The metadata mismatch worth checking

The reporter found three different size-related values:

hidden_size = 512
embedding_size = 512
text_config.embedding_dim = 768

One plausible explanation is that an output-dimension validator consulted a generic hidden-size or embedding-size field rather than the model-specific final output dimension. That is a diagnosis to verify in the code, not a confirmed maintainer root-cause statement.

The essential distinction is:

• Internal hidden size describes one part of the model architecture.
• Exposed embedding dimension describes the vector the embedding endpoint actually returns.
• Requested Matryoshka dimension specifies an intended output width supported by the model and runtime.

These values do not have to be interchangeable.

A minimal probe on an affected setup

The following reproducer mirrors the issue’s documented configuration. It needs a working vLLM installation, model access and supported GPU; it is not a result RAMGPT has executed.

from vllm import LLM, PoolingParams

llm = LLM(
    model="google/embeddinggemma-2",
    runner="pooling",
    dtype="bfloat16",
    enforce_eager=True,
    max_model_len=2048,
    gpu_memory_utilization=0.45,
    limit_mm_per_prompt={"image": 0, "audio": 0, "video": 0},
    hf_overrides={
        "is_matryoshka": True,
        "matryoshka_dimensions": [128, 256, 512, 768],
    },
)

prompts = ["What is artificial intelligence?"]

for dimension in (None, 512, 768):
    try:
        params = None if dimension is None else PoolingParams(dimensions=dimension)
        output = llm.embed(prompts) if params is None else llm.embed(
            prompts, pooling_params=params
        )
        print(dimension, len(output[0].outputs.embedding))
    except Exception as error:
        print(dimension, type(error).__name__, str(error))

If the third request fails before inference while the default request succeeds, capture the version, model metadata, override configuration and precise validation message.

The issue also mentions an unrelated Triton shared-memory initialization problem on the reporter’s RTX A5000. Do not assume a failure during model startup is the same as this dimensions-validation bug.

Production-safe workaround

For applications requesting the model’s native width, make the dimension option absent rather than explicitly passing 768 on an affected runtime. Then:

1. Confirm the returned vector length equals the index’s configured dimension.
2. Keep all vectors for a given searchable index at one compatible width.
3. Record model, model revision, runtime version, pooling configuration and vector dimensionality.
4. Run a small nearest-neighbour sanity check before swapping the index or embedding provider.

For consumers that require 512-dimensional vectors, keep dimensions=512 only if that width is supported by the checkpoint and results meet retrieval-quality requirements.

Do not solve this by adding zeros, cutting an arbitrary 768-vector to 512, or relabelling its metadata. A reduced embedding is only meaningful according to the model’s supported truncation scheme and downstream consistency requirements.

Why this matters beyond one model

Vector databases commonly create an index with a fixed number of dimensions. An incorrect default-versus-explicit validation rule can break migrations or rolling deployments even when the underlying model produces the right width.

An embedding API should distinguish “output vector shape” from “internal network hidden size.” A clean test compares the actual returned vector with the accepted requested dimensions, rather than reading one generic model configuration field.

The vLLM PoolingParams documentation describes the dimensions option as a way to reduce embedding width when the model supports it. The upstream bug report demonstrates why the supported range needs to reflect the model’s real output interface.

Status, October 17, 2026: issue #60977 remains open. The reporter proposed resolving the pooling output size from embedding_dim before falling back to hidden size; until that lands upstream, omitting dimensions for the native 768-wide output remains the least invasive workaround described in the issue.

Primary sources

• vLLM issue #60977 and recorded probe
• vLLM PoolingParams API documentation