Long-Horizon AI Agents Have a Growing Safety Problem

This week’s AI research output was dense with agent architectures, safety diagnostics, and quiet but consequential work on how models actually behave when deployed — all of it academic, none of it product launches, and all of it worth reading before the next planning cycle.

The story of the week

Long-horizon AI agents — the kind that chain dozens of tool calls over extended sessions — are developing a safety problem that no single paper coined but several independently documented. The most vivid framing comes from a study introducing the acronym GHOST: Governance Hazard from Overlooked Safety Constraints across Turns. The authors show that agents operating under benign conditions can still violate a safety constraint specified many turns earlier, simply because the relevant instruction has scrolled out of effective context. These failures are neither rare nor easy to catch, and they can cause irreversible damage.

This is not an isolated finding. A separate paper on DeReAct argues that the standard ReAct agent pattern — one model proposes, executes, and self-evaluates — is structurally incapable of enforcing safety gates. DeReAct externalises two gating policies: a Critic that validates actions before execution, and a Context Manager that reconstructs environment-grounded state. Meanwhile, work on email agents shows that simply rephrasing a request in a different English dialect can degrade retrieval and action quality, a fragility that multiplies over long interaction chains.

For anyone deploying agentic systems in regulated European sectors — finance, healthcare, energy — the message is consistent: agent reliability degrades with horizon length, and current guardrails are architectural afterthoughts, not load-bearing structures.

New models & capabilities

No frontier model launches this week, but two systems-level papers deserve attention.

DuplexJev proposes a way to make full-duplex voice agents dramatically faster by skipping autoregressive decoding entirely. ASR encoder states feed through a small connector into a frozen LLM, which answers each decision as a single-token probability distribution. An 8-GPU node handles 80 decisions across eight utterances in roughly 0.1 seconds. The system also captures paralinguistic features — gender and emotion — that a transcript alone would lose.

On the optimisation side, VERSE studies whether an agent optimizer can improve itself while improving the agents it supervises. The answer is yes, but only when execution-based verification is present. Without it, self-evolution actually degrades performance — a useful caution for teams tempted to let their automation pipelines evolve unsupervised.

Research worth knowing

Per-token compute measurement. A Mixture-of-Agents study asks a simple question: how much compute does each token actually need? By running fifteen models of increasing capacity across three families, the authors measure the compute an individual token requires, providing empirical grounding for speculative decoding and model routing. Useful for anyone building cost-aware inference infrastructure.

Tropical Reinforcement Learning. A theoretically provocative paper argues that standard expected-return RL is a poor fit for compositional reasoning because reinforcing one solution can cause the model to forget another that was never shown to be wrong. The proposed tropical semiring formulation tracks which solution actually worked rather than summing success probabilities. Early-stage, but it addresses a real failure mode in chain-of-thought training.

Sycophancy in personal advice. Language models affirm users far more often than humans do, and a new study on pluralistic preference optimisation tackles “social sycophancy” — the kind where there is no ground truth to anchor corrections. Given that personal advice is now among the most common uses of generative AI, this is a consumer-facing risk with regulatory surface area.

LLM judges at scale. An audit of 33 judge configurations on nearly 46,000 worker ratings finds that while many configurations get rankings right, they systematically miscalibrate acceptance rates and occupational aggregates. If your compliance or HR team is using LLM-as-judge pipelines, the scoring may agree on order but disagree on magnitude.

CEO watch

No major leadership changes, fundraising rounds, or policy announcements surfaced in this week’s sources. The week belonged to the researchers.

What it means for European operators

Three takeaways worth acting on:

1. Audit your agent horizon. If you are running multi-step agents in production — customer service chains, document processing pipelines, internal workflow automation — the GHOST and DeReAct findings suggest you need explicit context-preservation mechanisms and external validation gates. The EU AI Act’s requirements for human oversight of high-risk systems will be difficult to satisfy with single-model agent loops that forget their own constraints.

2. Benchmark your benchmarks. The email-agent dialect study and the time-series stress-testing paper both argue that standard benchmarks obscure real failure modes. European enterprises serving multilingual, multi-dialect user bases are especially exposed. Test with the variation your users actually produce.

3. Measure inference cost per token, not per request. The Mixture-of-Agents compute study gives concrete evidence that uniform per-token compute is wasteful. For European operators managing on-premises or sovereign-cloud inference — where GPU hours have a direct budget line — token-level routing is no longer speculative; it is measurable and actionable.