Open Problems in Agentic AI Security

Assume the Model Can Be Fooled

I came to security from vehicle safety engineering, where nobody certifies a car by assuming the driver never makes a mistake. You assume the mistake and design the vehicle so it is survivable. Agent security is settling on the same posture. A system that takes instructions in natural language can be talked into things, no amount of training will fully fix that, and the useful question is not whether an injection can succeed. It is what the injection reaches when it does.

That posture sorts every proposed control into one of three generations, and the generation tells you what the control actually guarantees.

Most agent security spend sits in the first two boxes. Keep them, but do not let them carry the system, because an adaptive attacker gets past both. The designs I build lead with the third: deterministic enforcement at the point where the action takes effect, so a fooled model still cannot reach anything that matters.

Before any detailed work I check three capabilities: whether the agent can read private data, whether it ingests untrusted content, and whether it can send anything out. All three at once, with nothing deterministic in between, is an exfiltration engine waiting for its prompt. Removing one leg is often the cheapest control you will ever buy.

The field’s own rankings are converging on this posture. OWASP’s 2026 Top 10 for LLM applications held prompt injection at number one even though, counted by raw public incident data alone, it would fall off the list entirely; OWASP reads that gap as a defense effect, teams spending hard enough that clean exploits rarely reach a public database. The larger signal is excessive agency, the polite name for handing an agent more privilege than its task needs, climbing to third, the biggest move on the list. The injection is rarely the expensive part. The privilege the agent was carrying when it hit is.

Where each control belongs and what your blast radius actually is depends on the control structure of your deployment: who can instruct the agent, what it can touch, and what watches it act. Mapping that structure is where every engagement starts, and it comes out differently every time.

Expert Fingerprinting in MoE Routing

Mixture-of-experts models route each token to a handful of specialists, invisibly. Latency and failure patterns leak that routing anyway. Probe long enough and you can map which expert handles what, then target the weakest safety behavior.

Mixture-of-experts models get their efficiency by not using most of the model most of the time. A router examines each token and dispatches it to a few specialist subnetworks out of the dozens or hundreds available. Which experts fire is an internal implementation detail: no API returns it, no response header exposes it, and most threat models treat it as invisible.

It leaks anyway. Different experts mean different computation paths, and different paths have different timing signatures. Failure modes leak too: a model degrades differently, refuses differently, and hedges differently depending on which specialists caught the request. Send enough structurally similar probes with small perturbations, correlate the responses, and the routing stops being secret.

The router’s choice never appears in a response. It shows up in latency and refusal patterns anyway, and enough probes turn those into a map.

An attacker who maps the routing has not extracted a single weight or broken a single control, and does not need to. Safety behavior is not uniform across a model; alignment training covers some regions of the input distribution more thoroughly than others. Once you know which prompt patterns land where, you can aim traffic at the paths where the safety training is thinnest. It is reconnaissance rather than exploit, which is why it falls off threat models: nothing was breached, nothing tripped an alert, and the whole campaign looks like a high volume of slightly odd queries.

Security has been here before. We spent two decades learning that shared hardware leaks: cache timing, power analysis, speculative execution. Shared model infrastructure is relearning the same lesson. Anything that changes the computation changes something observable, and an adversary with patience will find it.

What to do about it today is mostly posture. Providers should treat behavioral variance as telemetry and watch for probing patterns: large families of near-duplicate queries are a signature, not noise. Enterprises inherit this exposure from whichever provider they build on, so it belongs on the vendor-assessment list, and no control on your side should depend on the model’s internals staying secret. OWASP’s 2026 list now states the same principle for everything the model holds, under Hidden Context Exposure: nothing in the context should need to stay secret to keep you safe. Assume the architecture is enumerable and design as if the adversary has the map.

Identity Systems Built for People, Not Agents

Give a thousand agents scoped credentials and the directory quietly becomes the largest unmanaged identity surface in the company. Each holds real authority, and nobody has built a process for offboarding one.

Enterprise identity is built around a human lifecycle. Someone is hired, provisioned, given a multifactor token, reviewed in quarterly access attestations, and eventually deprovisioned when they leave. Every control in the stack quietly assumes that rhythm: human timescales, a manager who can vouch, a person on the other end of the authentication.

Agents break every one of those assumptions. They spawn in seconds and in thousands. They have no hire date, no manager to attest for them, and no natural end of employment. What they do have is authority: inherited from the user they act for, delegated downward through chains of sub-agents, exercised against real systems at machine speed.

The industry has a preview of how this goes: service accounts. They are already the worst-audited corner of every directory, full of passwords that never rotate, owners who left years ago, and permissions nobody can explain. That problem involved hundreds of identities. Agents make it tens of thousands, except these identities read mail, move money, and write to systems of record.

The delegation chain is the part that keeps me up. An agent acts on behalf of a person. It spawns a sub-agent, which calls a tool that acts with the original person’s authority, three hops from anything that authenticated. When that person changes roles, what happens at hop three? Today the honest answer at most organizations is: nothing. The credential still works.

Every hop carries the person’s authority. The kill switch only knows about the first one.

The frameworks are starting to price this in. Excessive agency just climbed to third on OWASP’s LLM Top 10, and identity is where excessive agency lives or dies: every standing credential, every over-broad scope, every delegation nobody can trace is privilege waiting for an injection to spend it.

What good looks like is knowable, even if almost nobody has built it. Issue credentials per task rather than per agent, short-lived enough that theft has a shelf life. Bind each agent’s identity to the delegating human, so authority dies when theirs does. Make expiry the default instead of standing access, and make delegation depth an explicit policy rather than an emergent property. Then treat offboarding an agent as a first-class, tested process, the way it is for a departing employee: the agent knew things and could do things, and both need to end on purpose.

The time to build this is now, while agent counts are small. The alternative is discovering the problem the way service accounts were discovered, in the middle of an incident.

Poisoned Context That Outlives the Session

A corrupted memory or retrieved document persists, shaping decisions days later with no obvious link back to the injection that caused them. That distance between where harm surfaces and where it entered is the whole problem.

Most prompt-injection discussion assumes a single-turn shape: malicious input arrives, the model does something bad, the session ends. Evaluation is built around that shape too: red-team the prompt, score the response, report the rate.

Persistence changes the shape. Agents keep memory. Retrieval pipelines index documents into vector stores. Context accumulates across sessions by design, because that accumulation is the feature. A poisoned document does not have to trigger anything on arrival. It sits in the store, gets retrieved next Tuesday, and nudges a decision that looks locally reasonable to everyone involved, including the agent.

Stored injections do their damage later, and later is what breaks the investigation.

Web security drew this exact distinction twenty years ago: reflected cross-site scripting versus stored. Stored was always the worse one, because it waits, and because it reaches users who never saw the injection happen. We are rebuilding that distinction around a new persistence layer, agent memory and retrieval stores, except this layer routinely gets less care than any production database even though an agent will act on what it reads there.

The operational problem is the distance. When harm surfaces days after the poisoning, nothing in the logs connects the bad decision to the document that shaped it. The trace shows an agent retrieving “context” and acting on it, all of it procedurally normal. The write that planted the poison looked like an ordinary write. Incident response becomes an exercise in reconstructing context lineage, and almost nobody logs enough to do it.

The defenses are unglamorous, which may be why they are rare. Tag every piece of context with provenance: where it came from, when, and how much it is trusted. Verify at write time, not just read time, so the store is defended at its front door. Expire memory so it ages out instead of accumulating indefinitely. Put change control on any store an agent reads before acting, with the same discipline a production database gets. And run evaluations across sessions rather than within one: inject in session one, measure in session ten. A test suite that only scores single turns cannot see this failure class at all.

Tool Provenance Is a Supply Chain

An agent’s capability surface is now effectively a package manifest. Every connected tool is code you did not write, executing with your users’ authority against your real systems, and the ecosystem ships far faster than any review process has kept up with.

Software already learned this lesson at cost. The npm compromises and the build-pipeline attacks taught the industry that you ship everything your dependencies ship, and it answered with lockfiles, signing, and software bills of materials. An agent’s tool manifest is the same object with higher stakes: third-party capability, wired directly to a system that acts with your users’ authority. OWASP’s 2026 rankings moved supply chain up the LLM list and widened the entry to cover model artifacts that are not what they claim to be, which is the same trust failure one layer down.

Two things make tools sharper than packages. First, a malicious library still has to find its way into a code path before it runs. A malicious tool just has to be connected; the agent will find reasons to call it. Second, a tool’s description is itself input to the model. The metadata is instructions. A tool that describes itself deceptively is not just misdocumented, it is actively steering the agent, which makes the manifest a prompt-injection surface before a single call is made.

The manifest was reviewed once. The tools behind it keep moving, which is why the runtime has to bound what any one of them can do.

Versioning makes it worse. A tool that was safe at review time is not the tool running today; remote tool servers update on the provider’s schedule, not yours, and nothing re-triggers your review when they do. The trust decision was made once, against a version that no longer exists.

Meanwhile the velocity mismatch is structural. Enterprise security review was built for quarterly vendor onboarding. The tool ecosystem ships daily. Any governance that requires a human meeting per tool has already lost the race, which is how organizations end up with capability surfaces nobody has actually read.

The response has two halves. The supply-chain half is familiar: pinned versions, allowlists, signed manifests, and review that covers tool descriptions as carefully as tool code, because the description is the part the model obeys. The architectural half is the honest admission that review will never fully keep up, so the runtime has to bound the damage: each tool scoped to the least agency it needs, returned data fenced so the model treats it as information rather than instruction, and blast radius per tool as an explicit design number. You cannot audit your way out of a fast-moving ecosystem. You can decide in advance how much any one compromised tool is allowed to cost you.