Pentesting an AI agent (also called an LLM agent in current shorthand) in 2026 means testing four layers a web application does not have: a planning layer that interprets natural language and decides actions, a tool layer where the agent calls external systems with privileges, a memory layer that persists context across runs, and non-deterministic behavior that produces different outputs for identical inputs. The methodology adds prompt injection at multiple input points, tool poisoning, memory poisoning, agent privilege escalation through tool chains, and hallucinated dependency abuse on top of standard web application testing. This post walks the threat model, the attack surface map, the test cases we run in our engagements, and the most common findings. For Series A SaaS founders building agentic features, this is what you should expect a pentest to cover.
Why AI agents need their own pentest methodology
A traditional web application has a request-response shape. The user sends an HTTP request, the server processes it, returns a response. Authentication, authorization, and input validation are well-understood. Pentest methodology for web apps is mature: OWASP Top 10, OWASP WSTG v5.0, PTES.
An AI agent breaks every assumption that methodology was built on:
- The user’s input is a natural language instruction, not a structured request. The agent interprets it, plans, and executes a sequence of actions. Authorization is harder when the action set is open-ended.
- The agent calls tools (APIs, databases, external systems) on behalf of the user. The tools have their own authentication and authorization. The agent is a privileged intermediary with broader access than any individual user.
- The agent has memory. Previous conversations, retrieved documents, and stored facts influence current behavior. An attacker who controls any input source eventually controls the agent.
- The model is non-deterministic. The same prompt may produce different outputs across runs. Reproducibility is harder. Coverage is harder.
These are not minor variations on web app testing. They are a different system. Pentesting an AI agent with web app methodology alone is like running a network scanner against a database server: you find some things, miss others, miss the ones that matter.
Attack surface map for AI agents
We use this four-layer model when scoping agent engagements:
Layer 1: User input (natural language)
↓
Layer 2: Planning (LLM decides what to do)
↓
Layer 3: Tool calls (external systems with privileges)
↓
Layer 4: Memory + state (persistence)Each layer has its own attack surface. Each layer can be the entry point. A vulnerability at any layer can compromise the agent end to end.
Layer 1: User input
Standard web application input validation applies (XSS, SQLi if input flows into a database, command injection if input flows into a shell). Plus prompt injection: an attacker provides input that the LLM treats as instructions instead of data. See our prompt injection 2026 patterns post for the full taxonomy.
Layer 2: Planning
The LLM receives user input plus system prompt, plus tool descriptions, plus retrieved context. It decides which tool to call with which arguments. Attack surface here:
- Indirect prompt injection through retrieved documents. The agent reads a document, the document contains an instruction, the agent follows it.
- Tool description poisoning. An attacker who can modify tool descriptions can change agent behavior without modifying user input.
- Plan manipulation. The agent can be tricked into multi-step plans that individually seem reasonable but together produce harm.
Layer 3: Tool calls
The agent invokes APIs, databases, file systems, web requests, code execution, email, payment, anything you have wired up. Attack surface here:
- Over-privileged agent. Most common finding. The agent has production tokens broader than any human user. A successful prompt injection inherits all of those privileges.
- Tool argument injection. The agent calls a tool with attacker-controlled arguments. If the tool does not validate (because it trusted the agent), classic injection vulnerabilities reappear.
- Tool chaining for privilege escalation. Tool A returns data the agent passes to Tool B. The combined effect exceeds either tool’s intended privilege.
- Side-effect tools. The agent calls a tool that writes to email, slack, payment systems, or production databases. Confirmation steps are often skipped under prompt-injection-induced reasoning.
Layer 4: Memory and state
Persistent context across sessions, RAG vector stores, conversation history, user preferences. Attack surface here:
- Memory poisoning. An attacker injects content into the agent’s memory that influences future sessions for other users.
- RAG poisoning. An attacker contaminates the document store the agent retrieves from. Future retrievals return the poisoned content.
- Cross-user memory leakage. Memory implementation accidentally shares context across user sessions.
- Memory exfiltration. The agent is induced to reveal stored memory contents in responses.
Test cases we run
The following test cases are run in every AI agent engagement. Specifics vary by agent architecture, tool set, and use case, but the categories are consistent.
1. Direct prompt injection at user input
Adversarial prompts at the user-facing input. Goals: bypass safety filters, extract system prompt, change agent persona, induce unauthorized tool calls. Tools: Garak, PyRIT, promptfoo, custom payloads.
2. Indirect prompt injection through retrieved content
We embed adversarial instructions in documents the agent retrieves: web pages, email content, support tickets, attached files. The agent reads, follows the embedded instruction, executes outside the user’s intent. This is the highest-severity finding pattern in 2026 because mitigation is structurally hard.
3. Tool poisoning
If the agent reads tool descriptions from a configurable source, we test whether modified tool descriptions change agent behavior. We also test whether new tools introduced at runtime are accepted without validation.
4. Tool argument injection
For each tool the agent can call, we test whether attacker-controlled prompt content can produce attacker-controlled tool arguments. Classic SQL injection, command injection, and SSRF patterns reappear here when tools trust agent-supplied arguments.
5. Tool chain privilege escalation
We map the agent’s tool graph. Identify tool combinations where Tool A’s output becomes Tool B’s input in ways that exceed either tool’s expected privilege. Test for unauthorized read-then-write patterns, cross-tenant data leakage, and information aggregation that violates least-privilege intent.
6. Memory and RAG poisoning
For agents with persistent memory or RAG, we test whether memory entries created by one user influence behavior for other users. We poison the document store with adversarial content and observe retrieval and downstream agent behavior.
7. Plan manipulation
Multi-turn conversations where each turn is benign but the cumulative state drives the agent toward a harmful action. The agent’s plan is the attack surface, not any single message.
8. Hallucinated dependency abuse
If the agent generates code that imports packages, we check whether non-existent package names are generated. An attacker who squats those names on npm or PyPI gains code execution if the generated code runs anywhere automated.
9. Output validation and information disclosure
The agent’s response is itself an attack surface. We test whether the agent reveals system prompt, internal tool descriptions, other users’ data from memory, or sensitive information from retrieval that the user should not see.
10. Authentication and authorization integration
Standard web application auth and authz tests applied to the underlying API endpoints, plus tests specific to agent context: does the agent enforce per-session authorization on tool calls, are there ways to escalate privileges by exploiting agent state, can the agent be tricked into bypassing auth checks intended for human users.
Most common findings (2026)
From our engagements over the past 12 months, the top three findings in AI agent pentests are:
- Over-privileged agents. Agent has production credentials with scopes broader than any human user.
- Indirect prompt injection through retrieved content. Tool outputs, retrieved documents, or external content steer the agent’s behavior.
- Insufficient output validation. Agent reveals system prompt, internal context, or other-user memory in responses.
The first two account for roughly 70 percent of high-severity findings. The fix patterns are well-understood but consistently missed in implementation.
How AI agent pentest fits into your security program
An AI agent pentest is not a replacement for traditional web application pentest. The underlying API surface, the authentication layer, the database, the cloud infrastructure all need standard testing. Agent-specific testing extends the standard methodology with the four layers above.
For a Series A SaaS startup with an AI agent feature in production, we recommend:
- First engagement: standard web app pentest plus agent-layer testing. Combined scope. Typically a Growth plan engagement at INR 1,79,999 plus one additional scope for agent depth at INR 74,999 = INR 2,54,998 total. 12 to 14 calendar days.
- Annual cadence: repeat the combined engagement annually as the agent’s tool set and memory architecture evolves. Agentic features change faster than typical web apps; the pentest cadence should reflect that.
- Continuous monitoring: for high-stakes agents (financial, healthcare, legal), add bug bounty or continuous testing through one of the AI-specific platforms. Static annual testing alone is insufficient.
See our AI Application Penetration Testing service page for scope details, or our pricing page for plan options.
Where to go from here
If you have an AI agent in production or in pre-production scoping and want to understand what a real pentest would cover, book a 30-min call with Ashok to scope the engagement. If you want a low-friction starting point, Security on Demand (INR 9,999, fully refundable) gives you four hours founder-led to map your agent’s attack surface and identify the highest-priority test areas before scoping a full pentest.
We work with AI-first and API-first SaaS startups, Seed to Series B, primarily based in Bengaluru.
Frequently asked questions
How is pentesting an AI agent different from pentesting a web application?
An AI agent has additional attack surfaces a web app does not: a planning layer that interprets natural language and decides actions, a tool layer where the agent calls external systems with privileges, a memory layer that persists context across runs, and non-deterministic behavior that means the same input may produce different outputs. Web app pentest covers OWASP Top 10. Agent pentest adds prompt injection at multiple layers, tool poisoning, memory poisoning, agent privilege escalation through tool chains, and hallucinated dependency abuse. The threat model is fundamentally larger.
What is the most common finding when pentesting AI agents?
Over-privileged agents. The agent is given production tokens, database write access, or API keys with broader scope than any human engineer would receive. The reasoning is convenience: it is easier to grant the agent broad access than to engineer least-privilege per task. The result is a single prompt injection becomes a production-wide blast radius. The fix is to scope agent privileges per session, per task, with audit logging on every tool call.
Can prompt injection be fully prevented?
No, not with current LLM technology. Prompt injection at the input layer can be mitigated with input validation, separator tokens, and content sanitization, but not eliminated. Indirect prompt injection through retrieved documents, tool outputs, or memory is harder to mitigate because the model treats those inputs as instructions. The realistic posture is: assume prompt injection will eventually succeed, design the agent so a successful injection cannot cause harm beyond a bounded blast radius.
What tools do you use to pentest AI agents?
We use a combination of public testing tools (Garak, PyRIT, promptfoo for adversarial prompt evaluation), custom prompt injection payload libraries we maintain internally, manual exploration based on the agent’s specific tool set, and traditional web application testing tools (Burp Suite, custom HTTP fuzzers) for the underlying API surfaces. The methodology is more important than the tools. Garak alone gives you a numeric score; finding actual exploits requires understanding what the agent is supposed to do and where it can be made to do something else.
How long does an AI agent pentest take compared to a web app pentest?
Typically 1.5 to 2x a comparable web application pentest, because the attack surface is larger. A standard web app pentest at Cyber Secify runs 7 calendar days. An AI agent pentest of equivalent complexity runs 10 to 14 calendar days, more if the agent has many tool integrations or persistent memory. We scope agent engagements based on tool count, memory architecture, and number of orchestration layers rather than just request count or endpoint count.