Heads of Security: Attention Gates Versus Prompt Injection

On October 27, 2026, researchers from Imperial College London, the Tübingen AI Center, ELLIS Institute, Max Planck Institute and Fraunhofer HHI released Heads of Security, or HoS. Its proposition is more specific than making a language model “safer”: teach a frozen model to read untrusted material without treating instructions inside that material as commands. Instead of retraining millions of weights, HoS learns small, per-attention-head gates that modify how trusted and untrusted token streams interact.

The paper’s adaptive-attack results are substantial, but the engineering question is what the numbers actually prove. HoS relies on a trust-channel assignment supplied by the surrounding application. It cannot decide whether an instruction was authorized, whether a tool result was mislabelled, or whether a legitimate workflow requires following an external instruction. A deployable defense therefore has two parts: the learned gate and the software that labels tokens correctly.

This is a review of the paper and its public implementation. The results below belong to the researchers. We did not download the weights, train gates, reproduce attacks, or benchmark inference.

The trust-boundary problem is not text classification

Consider an assistant asked to summarize an external webpage. The page contains useful facts plus a paragraph that impersonates the user’s instruction and asks the assistant to perform an unrelated action. The assistant must use the page’s facts, but the page is not authorized to issue new commands. Treating the whole page as harmless because it arrived through a legitimate retrieval tool is a category error.

A common defense is to add a system instruction warning the model to ignore malicious content. That changes the wording of the instruction hierarchy but leaves the untrusted bytes in the same inference computation. Another approach fine-tunes a model to prefer safe responses. It can improve robustness, but the adaptation may alter unrelated capabilities, especially when the training examples have a narrow format.

HoS instead encodes where each token came from. Its three channels represent trusted instructions and conversation, untrusted external content such as tool results, and chat-template formatting. For each attention head, a learned gate adjusts attention between query and key channels before the softmax. That means the defense can suppress the influence of untrusted instructions in some heads while preserving other heads’ ability to extract facts from the same source.

This is not a blanket deletion of untrusted content. A summarizer that cannot read its source is secure only in the trivial sense that it cannot do its job. The useful design target is a lower attack success rate at roughly unchanged legitimate task completion.

What was trained, and how small is it?

The authors evaluated GPT-OSS-20B and Llama-3.3-70B-Instruct. The underlying model weights remain frozen. The appendix describes 18,432 learned gate parameters for GPT-OSS-20B, including attention-sink gates, and 46,080 for Llama-3.3-70B. The GPT-OSS SecAlign comparison uses a 38-million-parameter LoRA adapter. Those figures describe trainable adaptation parameters, not the memory needed to load or serve the full base model.

Each GPT-OSS layer has a channel-pair-by-head gate tensor: 3 query channels × 3 key channels × 64 heads. Across 24 layers that is 13,824 entries, plus 4,608 sink-gate entries. The sum is 18,432. The Llama variant has 80 layers and no explicit sink gate, giving 80 × 3 × 3 × 64 = 46,080. This arithmetic is useful because it makes the claimed intervention inspectable; it does not by itself prove faster inference or lower end-to-end deployment cost.

Training uses Direct Preference Optimization on examples where following the user’s actual task is preferred over following instructions planted in data. The authors report eight H100 80GB GPUs for their distributed training setup. The repository README provides installation, gate-loading, tokenizer-channel and vLLM examples; its demonstration notebook was tested on an H100. That is a different hardware claim from saying that the gates themselves require eight H100s to serve.

A gate’s parameter count is not its full runtime cost. A serving stack must still propagate token-channel metadata through tokenization, batching, cache reuse and attention kernels. A benchmark that compares only checkpoint sizes misses these integration costs.

Reading the attack results without turning them into a slogan

On the held-out InjecAgent goals, the paper reports the strongest black-box RL-Hammer attacks reaching 97% attack success rate on undefended GPT-OSS-20B, 85% on SecAlign, and 10% on HoS. For the Llama-3.3-70B family, the corresponding maxima are 99%, 82%, and 1%. These are highest observed ASRs across the attack configurations evaluated, not a universal success probability for an arbitrary malicious webpage. The attacker’s training target, evaluation target, reasoning effort, success criteria and test split matter.

The white-box Claudini experiments are also instructive. The researchers optimized an attack specifically against HoS, then measured the cumulative compute required to reach 50% ASR on selected AgentDojo and InjecAgent tasks. They report roughly 2.6–3.6 times the attack compute relative to the compared undefended and SecAlign targets. That is a meaningful shift in attack economics under a controlled budget, not a proof that the defense cannot be broken.

Utility is the second half of the story. On five-shot MMLU-Pro, the paper reports GPT-OSS-20B at 68%, HoS at 67%, SecAlign at 59%, and ASIDE at 52%. On the Llama family, the paper reports 56% for the base model, 55% for HoS and 16% for SecAlign. The authors also report that their ASIDE baseline frequently failed to issue usable tool calls on agent benchmarks. This matters because a “defense” that stops the assistant from using tools can produce a superficially low attack rate while failing the user.

None of these comparisons should be generalized beyond the specified versions, datasets and adaptations. The paper is a preprint; results should be treated as author-reported until independently reproduced.

The missing control plane: who assigns the channel?

The most important deployment question is outside the learned gate. The model receives channel labels, but the application decides them. The paper explicitly states that HoS does not determine which sources are trusted or which instructions are authorized.

A safe integration would mark retrieved pages, search snippets, email bodies, repository README files and third-party API responses as untrusted by provenance, before their text is mixed into the model prompt. It should not infer trust from a phrase such as “SYSTEM:” inside the retrieved text. Nor should a downstream agent be allowed to relabel content as trusted simply because an upstream agent summarized it.

This introduces a precise failure mode: channel laundering. Suppose an untrusted webpage is summarized by one agent and the summary is inserted into another agent’s conversation as a privileged instruction. The first model’s gate may have worked perfectly; the second agent has already lost the origin metadata. HoS cannot repair that. Preserving provenance across tool calls and agent handoffs is therefore a security requirement, not an optional optimization.

Another ambiguity is legitimate delegation. If the user explicitly asks an assistant to follow installation instructions from a trusted package maintainer, blindly suppressing every instruction in the retrieved document could make the task impossible. The paper acknowledges that strict instruction–data separation is suitable when external content is intended as data, not whenever an application intentionally delegates authority.

A falsifiable reproduction and ablation plan

A meaningful independent reproduction should begin with the released model gates and code, pinning model revisions, package versions, dataset splits and the token-channel tokenizer. Use a clean test suite in which a retrieved document is needed for the legitimate answer, then inject an unrelated request into that document. The test harness should record both whether the unauthorized action occurred and whether the legitimate answer was correct.

Compare at least four conditions under identical decoding and tool permissions: the base model; the base model plus a textual warning; the published HoS gates; and HoS with intentionally incorrect channel assignment. The fourth condition is essential. If changing only the provenance label erases the advantage, the integration’s trust classification is part of the effective defense.

A second ablation should preserve the same attack set while changing the content’s format: ordinary web text, quoted email, Markdown tables, JSON tool results, and long documents with the injection buried far from the start. Do not allow the attacker to change the evaluation metric or the user task. Report task completion, attack success, refusal rate, false positives and tool-call accuracy separately.

For adaptive evaluation, use matched attacker budgets and report how many optimization attempts were allowed. A static attack library tests known payloads; it does not establish robustness to an attacker who knows the defense. The paper’s adaptive-attack results make this distinction central.

For serving cost, measure tokens per second, time to first token, GPU memory, batching throughput and KV-cache reuse with identical context length and concurrency. Test a request containing multiple tool outputs with different channel labels, then reuse the prefix cache across requests. If a cache key omits trust-channel metadata, identical token IDs with different trust labels may no longer be semantically equivalent. That is a hypothesis to test, not a demonstrated flaw in the released code.

Operational boundary: gates are a layer, not a permission system

HoS offers an unusually compact way to alter instruction-following behavior while retaining useful reading capability. Its strongest contribution may be the combination of constrained trainable parameters and a security–utility evaluation that includes adaptive attacks, few-shot tasks and agent tool use.

But deployment should still enforce authorization outside the model: scoped tool credentials, confirmation before high-impact actions, origin-aware data handling, audit trails and a deterministic policy for what the assistant may execute. An attention gate should never be the sole reason a tool is allowed to transfer data or change state.

The engineering conclusion is narrower and more valuable than “prompt injection solved.” A model can be trained to respect an explicit data/instruction boundary with surprisingly few parameters; the application must preserve that boundary end to end. That is a testable architectural hypothesis worth independent replication.

Primary evidence and limitations

The October 27 research paper supplies the mechanism, ablations, attack matrices and author-reported results. The public code and README document the released implementation, supported models, channel-aware tokenization and reproduction path. The released gate collection provides the model artifacts. No independent evaluation was performed for this article; all numerical performance and robustness claims above are attributed to the researchers.