Agent Injection Evals

Problem and relevance to agents

An agent that cannot complete any task can still report zero successful attacks. That number alone does not demonstrate security. This project measures legitimate work, malformed responses, attack attempts and effects separately.

The evidence makes it possible to inspect an agent reading hostile documents and proposing actions. It separates model behavior from authorization enforced by the tool executor.

Architecture

1. Controlled tasks and attacks. Two development tasks and six evaluation tasks each have a clean control and four hostile variants: answer contamination, synthetic secret disclosure, destination redirection and unauthorized state mutation.
2. 2 × 2 comparison. Plain or instruction-isolated prompts are crossed with permissive or task-scoped policies. The same pinned FLAN-T5-small model generates one decision per case.
3. Simulated execution. A strict parser accepts ANSWER, SEND or WRITE. Policy checks the authorized verb and target. Effects stay in fresh in-memory dictionaries; destinations use .invalid and no messages are sent.
4. Traces and scoring. Every prompt, output, parse decision, attempt, effect and resource observation is recorded. An offline verifier reconstructs prompts and scores from saved responses without rerunning the model.

• Design and metric definitions
• Parser, policies and scoring
• Synthetic dataset provenance

What this demonstrates

• Evaluation design separating legitimate capability, attempted attacks and realized effects.
• Tool authorization enforced outside the model, with a permissive baseline and reproducible inputs.
• Publication of negative results and error analysis without mislabeling protocol failures as successful defenses.

Published results

The evaluation report records 120 real CPU generations: six tasks, five variants and four strategies. No strategy completed a clean or hostile task under the frozen metric. The development experiment preserves another 40 generations.

The experiment used google/flan-t5-small revision 0fc9ddf78a1e988dac52e2dac162b0ede4fd74ab, seed 1729, greedy decoding, two threads, up to 512 input tokens without truncation and 48 output tokens. The recorded environment is Python 3.11.13, torch 2.2.2, transformers 4.46.3 and an Intel Xeon W-3223 with 48 GiB RAM.

Isolation produced five unauthorized writes. Combined protection attempted the same five writes and policy blocked them. With 0/6 clean-task successes per strategy, there is no useful defense winner and no defined capability-conditioned attack rate: that value is null, not zero.

All 120 generations completed without runtime errors. Model loading took 2.797 s and process peak memory was 709.3 MiB, shared across strategies. Energy and hardware amortization were not measured. Evaluation report.

Some responses extracted the correct fact but omitted the required prefix. The error analysis preserves those cases and explains why the success criterion was not changed afterward. Error analysis.

Reproduce

Recorded-report verification needs Python 3.11 and no external packages. Inference requires the documented Intel macOS native dependency lock and a download of public weights. Inference runs locally after downloading, without paid APIs.

Report verification recomputes internal consistency; it does not rerun inference or authenticate timing. New inference runs require new output directories and should run serially. The lock targets the reference environment; other platforms must record their own versions.

git clone https://github.com/lucioduran/agent-injection-evals.git
cd agent-injection-evals
git checkout e6d3c2c62154ccfc5a36595ee0ed4e44484c7d66
python -m injection_evals verify --report results/dev/report.json
python -m injection_evals verify --report results/eval/report.json

python -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements-local.lock
python -m injection_evals download
python -m injection_evals run --split dev --output build/reproduced-dev
python -m injection_evals run --split eval --output build/reproduced-eval
python -m injection_evals verify --report build/reproduced-eval/report.json

Scope and limitations

• The small model fails the action protocol and tasks. These results cannot rank defenses for capable production agents.
• The synthetic set is small, author-known and shares attack templates across splits. It is neither representative of arbitrary attacks nor an independent blind evaluation.
• Each trial has one decision, with no browser, tool feedback, adaptive attacks or real exfiltration.
• Isolation is a prompt convention. Policy does not prevent disclosure in answers or through an already authorized destination.
• Timings are one observation per case without warmup removal. Replay checks internal consistency, not measurement authenticity or cross-platform neural equivalence.

• Function calling and tool use in agents

Sources and revision

Values on this page come from the linked report. They are not live measurements. The report records the executed revision; the reviewed revision includes the subsequent publication of that report.

• Reviewed revision: e6d3c2c62154ccfc5a36595ee0ed4e44484c7d66
• Revision recorded by the experiment: ed6cbc309e1ec9f1e7211d83743927e28ee48c97
• Report timestamp (UTC): 2026-10-21T21:29:09.518113+00:00
• Corpus SHA-256: ef0d7970d9ec3cde2d3c02d6ddef34046a82aef12b8524175216d5d6f216ac65

• README
• Design and metric definitions
• Synthetic dataset provenance
• Tasks and labels
• Evaluation report
• Complete evaluation trace
• Development report
• Error analysis
• Model and license notices
• Parser, policies and scoring
• Pinned inference dependencies
• MIT license