ToxicBench: When Tools Silently Lie, Checking Is Not Enough

The tool call succeeds. The number looks plausible. The report still attaches top revenue to the wrong store—because the source table never moved; only the returned observation was quietly rewritten. On this site, SINGED asks whether a correct artifact implies a safe execution path; FTA asks whether agents still report success after tools have already failed. This post fills a third gap: when execution succeeds but evidence is poisoned, do agents check—and after checking, do they still adopt the wrong answer?

Zifu Tao and Changqing Yin (Tongji) introduce this setting as silent tool poisoning in arXiv:2609.37153 (When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents): a proxy rewrites the observation returned to the agent after the tool truly executes, while source tables stay fixed. Their benchmark ToxicBench and code at Toxic_Tool_Bench pair clean and poisoned environments under numerical, label, schema, and retrieval errors, and use trajectory metrics to separate adoption without checking from adoption after a qualifying check.[1]

Name collision first: another ToxicBench in the community targets NSFW image safety. This post is only about the data-agent silent tool-poisoning benchmark, unrelated to image moderation. It is a mechanism and evaluation piece—how observations are corrupted, how metrics split, how to read the numbers, and what to copy into a harness. No attack recipes; no how-to for poisoning tools.

How this lines up with SINGED and FTA

Three posts target three different kinds of “looks successful”:

Site entry	Surface	Real gap
FTA	User-facing “done”	Tool failed / evidence missing—is the report honest?
SINGED	Artifact passes the task oracle	Did execution trigger contract-forbidden process effects?
This ToxicBench post	Tool succeeded; observation looks well-formed	Was the observation silently rewritten? After checking, is the poisoned conclusion still adopted?

On tool boundaries, MCP Toolbox / tool-path SSRF governs egress and path reach; ToxicBench assumes the tool already finished successfully—the question is whether returned observations match source facts. This is also not indirect prompt injection: the paper keeps the analytical objective unchanged and does not ask the agent to follow malicious instructions; only evidence is corrupted. It is not “dirty source data” either: tables stay fixed, so the clean answer remains recoverable by checking, and a wrong answer cannot be excused as faithful analysis of altered tables.[1]

For data/analytics agents, treat the three posts as three gates on one acceptance axis: failure transparency → process-side cleanliness → observation-side verifiability. Miss a gate and end-to-end green can still be false green.

Silent poisoning: tables fixed, observations rewritten

Figure 1’s revenue example is concrete: the clean CSV tool returns Store B ($450,000); in the poisoned environment the proxy swaps labels and returns Store A ($450,000)—successful interface, legal format, plausible magnitude. Restoring the correct binding requires re-inspecting source rows, not checking that the amount is positive. Similar failures include aggregates with the wrong denominator, dictionaries that supply outdated column meanings, and retrieval snippets that look relevant but support the wrong conclusion—normal-looking tool output is enough to produce confident, incorrect analysis.[1]

Formally: query (q), tool (f) on input (x) yields (o=f(x)); the poisoned environment returns (\tilde{o}=P(f,x,o;\theta)) with source data unchanged. Silence means (\tilde{o}) preserves the expected interface, reports successful execution, and stays relevant enough to support a plausible wrong answer. Pairing the same agent, task, data, and model in clean and poisoned environments links the changed observation to later checks and final adoption. The proxy executes the tool first, then edits the returned observation; both versions are logged, but only the returned one is exposed to the model.[1]

Operators fall into two families. Numerical: aggregate scaling, sign flip, rank/label swap; the expanded suite adds ratio inversion, denominator swap, omitted-filter corruption, and unit conversion. Semantic/schema: entity label swap, treatment/control flip, column-semantic swap, stale metadata, biased retrieval. Magnitude checks catch some numerical errors; label swaps keep the amount and change the entity, so magnitude-only checks miss them—which is why “recompute the mean” is not always enough later.[1]

Tasks use small synthetic CSVs (sales, campaigns, inventory, clinics) plus dictionaries and evidence snippets; numerical tables have about 4–12 rows (median 5.5), semantic tables 3–6 (median 3.5). Queries cover aggregates, rates, rankings, or evidence-backed entity/field choices. After reference auditing and exclusions, the main evaluation keeps 118 single-table instances (58 numerical + 60 semantic/schema), plus a 13-task multi-table extension for discovery and joins. Model-free acceptance covers all 131 retained tasks: standard-library and pandas implementations agree with accepted answers; under fixed calls and rendering, 119 tasks receive an unambiguous conclusion-changing return, and all 119 admit a scripted raw-data recovery route under one-shot poisoning. This is mechanism coverage, not a production traffic sample; the limitations section notes small synthetic tables plus controlled public-table checks, with generalization to naturally occurring errors untested.[1]

Metrics: split checking from adoption

Final correctness alone cannot tell apart “trusted the first observation,” “checked against further bad evidence,” and “saw a correction but selected the wrong conclusion.” Counting extra tool calls is also insufficient: an agent may repeat the same computation, scan unrelated rows, or re-query the same unreliable source. ToxicBench therefore stacks trajectory metrics—TSR over all retained runs; behavioral rates conditioned on runs actually exposed to poisoning—and uses PDR to separate “was corruption delivered?” from “how did the agent respond once exposed?”:[1]

Metric	Meaning (paper)
TSR	Task success vs the clean reference
ΔTSR	Clean TSR − poisoned TSR
PDR	Fraction of toxic runs with at least one actual modification
ADR	Final response explicitly flags conflict, implausibility, or possible corruption
VR	After the poisoned observation, a fresh, task-relevant, evidence-producing tool event
RR	After ADR or VR, the final answer still matches the clean reference
PAR	Final answer matches the poisoned oracle (regardless of checking)
BCR	Adopts the poisoned conclusion with neither ADR nor VR (blind compliance)
VPA	Qualifying VR yet still adopts the poisoned conclusion (adoption after checking)

The key sentence: VR rules out blind compliance for that run, but does not by itself show recovery. VPA is exactly “checked, still chose poison”—especially under repeated poisoning, when follow-up checks can themselves be corrupted. After stripping adapter display labels, the scorer extracts conclusions tied to the requested answer role before matching clean and poisoned references; conflicting and unresolved selections stay in the denominator. Generic “may need verification” counts as neither ADR nor VR; you need a specific anomaly claim or a new evidence event.[1]

The paper also separates three kinds of independence: observation independence (not copying the poisoned return), execution-route independence (a fresh evidence-producing action), and source independence (different upstream tables, caches, or authorities). VR requires only the first two. In construction, the 120 original expanded tasks plus 13 join tasks reuse their source CSVs, so source-independent coverage is 0—a successful check can still share an upstream error. In deployment acceptance, write those three layers separately; do not treat “called a tool again” as “switched authority.”[1]

Main result: strong when clean, collapses when poisoned

In the 118-task GPT evaluation (Table 1, historical injector), clean TSR is high across three adapters and poisoned performance collapses:

Adapter	Clean TSR	Poisoned TSR	ΔTSR	BCR	RR	PDR
LangGraph ReAct	0.99	0.67	0.32	0.38	0.33	0.86
smolagents	0.98	0.59	0.39	0.53	0.15	0.74
AutoGen	0.92	0.66	0.26	0.27	0.32	0.86

Abstract wording: poisoning lowers task success by 26–39 percentage points (clean about 0.92–0.99 → poisoned about 0.59–0.67). BCR is nonzero on all three adapters—adoption without detection or checking is not a single-framework quirk. smolagents shows the highest BCR (0.53) and lowest RR (0.15); AutoGen has the smallest ΔTSR (0.26) and lowest BCR (0.27). Conditional rates describe exposed behavior, not a ranking on a common exposed set; adapters also differ in how often they call poison-eligible tools (PDR 0.74–0.86).[1]

The cross-model matrix (Table 2) runs GPT / Claude / Qwen × three adapters on 34 numerical + 24 semantic/schema instances: all nine fully crossed configurations show numerical TSR declines, and eight show semantic/schema declines; Claude–AutoGen shows a small semantic increase (clean 0.83 → poisoned 0.88). Vulnerability is widespread but varies by model and task family—do not read one cell as “immune.” Operator profiles (Figure 3) reinforce the point: recomputing aggregates can expose some numerical errors, but label swaps preserve amounts while changing entities, defeating magnitude-only checks.[1]

Another attempt ≠ verification; checking ≠ changing the answer

If a second execution can still obtain clean evidence, gains may come from another chance to solve the task rather than from a verification instruction. The paper therefore compares four protocols under shared caps (ten steps per route, two routes, 3,072 output tokens per request, temperature 0) with Claude Haiku 4.5 on 118 tasks:[1]

• Base: one ordinary route
• Double-pass: two routes with the same query; select the second answer; the second route does not see the first answer
• Verification-only: hand the first answer to the second route as an untrusted claim to check against source data; select the second
• Generic Guard: add a fixed expectation checklist to both routes (ranges, filters, units, denominators, entity–value bindings, column meanings, timestamps, source support), require a fresh tool action and literal labels; the wrapper returns the verification-route answer directly

Results (Figure 4): poisoned TSR is 1.00 for Double-pass, 0.89 for Base, 0.90 for Verification-only, and 0.97 for Generic Guard. Double-pass’s gain over Base is 0.110 (template-cluster 95% bootstrap interval [0.056, 0.171]). All three two-route methods drive BCR to zero, yet Verification-only retains nonzero PAR and VPA—eliminating blind adoption does not eliminate adoption after checking. Guard minus Double-pass poisoned TSR is −0.025 (interval [−0.054, 0.000]): in this comparison, explicit verification prompts do not stably beat an ordinary second attempt. On AutoGen the two verification prompts barely separate either: Guard clean/poisoned TSR 0.88/0.87 versus Verification-only 0.91/0.86.[1]

One explanation is evidence availability: one-shot poisoning state is shared across routes, so after the first corruption, later calls can receive unmodified observations. Double-pass combines another attempt with access to later evidence; its gain does not isolate verification reasoning. Latency rises too: on retained toxic runs Generic Guard averages about 39.79s, Base 16.91s, Double-pass 35.89s, Verification-only 33.62s (matched caps, not matched realized compute or token billing).[1]

Qualitative cases (§4.5) pin metrics to concrete actions. On a treatment/control task, the poisoned observation reports “Control–0.145”—correct value, wrong group. The base trajectory repeats it without checking rows; the guarded trajectory re-inspects the dataframe and returns “Treatment–0.145.” A magnitude check would miss this; recovery requires restoring the group–value binding. In a multi-table case, the guarded agent reconstructs the join before finalizing and checks which rows support the aggregate. Generic “may need verification” without new evidence earns neither VR nor RR.[1]

Public-table control: second-stage evidence nearly decides recovery

To test supplied evidence directly, the paper runs a field-bound control on 24 tasks across three public tables (Palmer Penguins, Auto MPG, Bike Sharing). First-stage correctness is 24/24 with clean evidence, 20/24 with partial alias corruption, and 0/24 with all target aliases corrupted. Holding a fixed poisoned first-stage transcript, two second-stage repetitions per task yield 48/48 correct retries and 47/48 correct reviews with fresh clean evidence, versus 0/48 for either method under full-target corruption.[1]

Read this carefully: it isolates a local evidence-condition effect; it does not prove verification prompts beat retry, and it does not explain the entire historical Double-pass gain. For harness design it is blunt—if the second stage still feeds the same poisoned observation, another “please check again” barely helps. Partial corruption (altering only the numeric alias while leaving comma-formatted clean copies) can also make resistance look higher than it is; full-target corruption regenerates all three aliases. Internal red teams should not treat half-poisoned observations as a pass line.

Repeated poisoning: adoption after checking (VPA)

Repeated poisoning lets every eligible observation be corrupted (probabilities 0.25–1.00). Across 1,572 retained trajectories (58 numerical + 60 semantic + 13 join × three two-route protocols × four probabilities), the scorer finds 101 VPA cases (all retained after excluding flagged delivery events); 78 have only corrupted qualifying follow-up checks, and 23 include at least one unmodified check—only one of the latter has an automatically identifiable clean-reference match, still needing evidence-level review.[1]

This establishes that adoption after checking is observable in traces; it does not claim all 101 agents ignored sufficient correct evidence. When routes share sources and backends, another check need not supply independent corrective evidence. At (p=1), all nine cells can show zero detected BCR while PAR and VPA remain nonzero—“no longer blind” can coexist with “still adopting poison.”

Human evaluation on a post-freeze packet of 200 trajectories (20 task IDs disjoint from development) agrees with automatic TSR on 192/200 (96%), precision 1.000 and recall 0.957; two annotators agree on every trajectory across six binary fields. On 77 exposed trajectories, humans confirm all four VPA cases—adoption after checking is not a scorer artifact. On the same 20 core tasks, humans also preserve Double-pass’s poisoned-TSR advantage over Base (1.00 vs 0.80).[1]

Neighboring work: which layer ToxicBench owns

Tool-calling benchmarks often score API selection and arguments; interactive and data-agent benchmarks score task completion. TRACE-style work uses evidence banks and LLM judges for trajectory efficiency, hallucination, and adaptivity. ToxicBench shares that process lens but additionally separates answers supported by returned observations from answers correct according to source data, tracking checking, adoption, and recovery separately.[1]

Silent tool errors appear in Tools Fail; broader tool-environment unreliability and recovery in ToolBench-X and PALADIN; stage-aligned perturbations in ToolRobustBench. Tool-metadata and retrieval poisoning (MCP-ITP, PoisonedRAG, SafeRAG, and others) are a different threat surface. In financial agents, prior work already finds that self-verification or contamination detection need not improve recommendations under manipulated tool data. ToxicBench’s contribution is to package these failures as paired data-analysis tasks + fixed source tables + separate checking/adoption/recovery metrics, with controls under one-shot and repeated corruption—not another fuzzy “tools can fail” score.[1]

Aligned with this site’s narrative, a five-layer map helps avoid topic collisions:

Layer	Question	Site / neighbor
Reporting	Are post-failure claims faithful?	FTA
Process	When outputs are correct, are side effects in bounds?	SINGED
Observation	Are successful returns still trustworthy; does checking change the answer?	ToxicBench
Path / egress	Can tools reach URLs/files they should not?	MCP Toolbox, OpenShell
Control flow	Are skill clauses mechanically fulfilled?	HEXIS

ToxicBench does not replace path or process layers. It forces a narrower admission: after the tool turns green, you can still fail at evidence selection.

Scoring and human checks: why freeze the scorer

If you port similar metrics in-house, copy the appendix’s process, not only the acronyms. The scorer first checks recorded poison delivery and strips adapter display labels; a frozen parser extracts query-related claims, explicit conclusions, corrections, and numerator/denominator roles, then matches references with categorical boundaries and complete-number matching (including thousands separators). Conflicting selections stay unmatched; unresolved claims stay in denominators—otherwise a method that “looks cleaner” may only be better at confusing the parser.[1]

A 120-trajectory development packet and a later 240-trajectory holdout informed parser repairs; the post-freeze 200 is the independent check: task IDs disjoint from development, annotators blind to automatic predictions. The authors keep the scorer fixed at the pre-annotation version and do not tune the reported parser on these 200 labels. That workflow matters internally—once you both repair the scorer and rank methods on the same traces, human agreement rates inflate. ToxicBench releases versioned scoring with trajectories so later readers can see which cut moved which number.[1]

One more reporting trap: TSR includes undelivered runs; behavioral rates use exposed runs only. An adapter that rarely calls poison-eligible tools may look better on poisoned TSR while still showing high conditional BCR. Report PDR beside behavioral rates; a single “robustness score” misleads stakeholders.

What to copy into a harness—not another “please verify” line

Read the numbers as five engineering moves:

1. Treat observations as untrusted inputs. Tool success ≠ trustworthy evidence. For ranking, binding, and denominator-sensitive conclusions, require a trail back to source rows or a reconstructed join—not only a restated aggregate. The “looks like verification” idle loops in cost-inefficient coding-agent behaviors are the same disease as verbal doubt without new evidence here.

2. Measure separately: did it check, and what did it adopt afterward? If internal evals only report poisoned TSR, BCR and VPA collapse into one blob. Split at least delivery (PDR), blind compliance (BCR), adoption after checking (VPA), and recovery (RR). Otherwise a verification prompt can look great under one-shot poisoning when later evidence is still clean.

3. Spend verification budget on independent evidence, not same-channel replay. Under one-shot poisoning, ordinary retry is already strong; under repeated corruption or shared upstreams, switch tools, switch aggregation paths (e.g., sum+count to audit a mean), or read raw rows. Generic Guard’s checklist is a useful policy draft, but the contrast shows no stable advantage over Double-pass from the prompt alone—advantage tracks what the second route observes.

4. Make answer-selection policy explicit. Verification-only hands the first answer to the second route; that can focus a check or anchor an error. Always selecting the second route can correct poison or override a correct first answer—use clean TSR for net effect and audit false overrides separately. A “verifier role” is not automatically safer.

5. Connect to supply-chain / MCP narratives without mixing threat models. ToxicBench corrupts returned observations, not whole-database poisoning, system-prompt hijacks, or tool-metadata poisoning (those live on other lines such as MCP-ITP and TRUSTDESC). Source-independent checks have coverage 0 in the benchmark construction; if production has proxies, materialized views, or shared caches, design observation independence and source independence separately. Credential and egress boundaries still belong to OpenShell / Sentry and the MCP path post—ToxicBench does not replace those controls.

If you make only one minimal change, prefer this: for entity–value binding conclusions, require a source-row evidence edge—any store/group/column name in the final answer must point to a field from a raw-row or join-reconstruction event, not only to one summary line from an aggregate tool. ToxicBench’s qualitative cases repeat the same lesson: the number can be exactly right while the binding is wrong; without that edge, magnitude checks and “please verify again” can idle. That is usually cheaper than a full Double-pass and more controllable than hoping a Generic Guard prompt grows independence on its own.

Limitations and reading traps

• Core tasks use small synthetic tables; public-table controls broaden the data setting but keep seeded calls and structured answers.
• Scorer recall varies on fine metrics (development audits show low ADR/BCR recall); read method rankings with human comparisons.
• The historical injector has delivery exceptions such as sign-flip (204 flagged events among 3,319 recorded corruptions); the appendix reports sensitivity; the main-text 118 are the retained set after reference revision.
• The historical PandasAI adapter edits the final answer only after the agent finishes, so the agent cannot check—it is excluded from behavioral comparisons; do not misread that as “safer framework.”
• DA-Agent completed only the GPT numerical suite; cross-model means are unweighted averages over the three common adapters, not a pooled trajectory pool.[1]

Closing

ToxicBench turns “can tools silently lie?” from a slogan into a paired, trajectory-splittable evaluation: source tables fixed, observations corruptible; successful execution no longer equals trustworthy evidence. The 26–39pp TSR collapse on 118 GPT tasks, nonzero BCR, and VPA under repeated poisoning jointly say that checking and adoption are two dimensions—nudging agents to “look again” is not enough; you also have to ask what the second look saw and which answer was finally selected. Lined up with SINGED and FTA on this site, data-agent reliability at least needs three gates at once: honesty after failure, clean paths when outputs are correct, and skepticism toward successful observations.

Code and trajectories: Toxic_Tool_Bench; paper HTML: arXiv:2609.37153.[1]

References

[1] Zifu Tao, Changqing Yin. When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents. arXiv:2609.37153, 2026. abs · HTML · code