AI Agent Security: The Developer's Guide to Runtime Protection
1 in 7 agent sessions contains an injection attempt. Here's how to detect, block, and respond to threats at the I/O layer — in 3 lines of Python.
from runesec import Rune
rune = Rune()
result = rune.scan(user_input)What is AI agent security?
AI agent security is the practice of protecting autonomous AI systems — agents that take actions, call tools, and process data on behalf of users — from threats that exploit their expanded capabilities. Unlike traditional application security, which focuses on known inputs and predictable control flows, agent security must handle dynamic, unpredictable interactions where a single malicious prompt can cause an agent to exfiltrate data, escalate privileges, or execute unauthorized tool calls.
There are two architectural approaches to agent security. AI Security Posture Management (AI-SPM) focuses on discovering and governing agents across an organisation: inventory, policy, compliance posture. Agent Detection and Response (ADR) focuses on runtime protection — scanning every input, output, and tool call in real time to detect and block active threats. Both matter. This guide focuses on the runtime side: what happens when your agent is live and under attack.
Why AI agents need runtime security
AI agents are not chatbots with extra steps. They have tool access, persistent memory, and the ability to take actions with real consequences — sending emails, executing code, querying databases, modifying files. That expanded surface area creates attack vectors that don't exist in traditional LLM applications.
The numbers are stark:
- 14.2% injection rate — Rune's production data shows roughly 1 in 7 agent sessions contains an injection attempt. This isn't theoretical. It's what's happening in production right now.
- 340% year-over-year increase in prompt injection attacks, per OWASP's 2026 LLM Security Report — the single fastest-growing category of cyberattack globally.
- 80% success rate demonstrated in a January 2026 study where a single poisoned email coerced GPT-4o into executing malicious Python that exfiltrated SSH keys.
The core problem: agents operate in an environment where inputs are untrusted, outputs have real-world consequences, and the attack surface grows with every tool you connect. Traditional security tools don't see this layer. WAFs can't inspect prompt payloads. SIEMs don't understand tool call semantics. Endpoint protection doesn't know what a valid agent action looks like.
You need security that understands agent architecture — at the I/O layer, in real time, at the speed your agent operates.
What are AI agents?
Before securing agents, it helps to understand what makes them different from standard LLM applications.
AI agents are systems that use large language models to reason about tasks, make decisions, and take actions autonomously. They typically combine an LLM with tool access (APIs, databases, file systems), memory (conversation history, vector stores), and planning capabilities (multi-step task decomposition).
Common agent architectures in production today include simple reflex agents that respond to patterns, model-based agents that maintain internal state, goal-based agents that plan toward objectives, and learning agents that adapt from feedback. The security implications scale with autonomy — a simple Q&A agent has a limited blast radius, but an agent with code execution, database access, and email capabilities can cause significant damage if compromised.
Agents are being deployed across customer support (Salesforce Agentforce, Microsoft Copilot Studio), internal workflows (ServiceNow, custom LangChain agents), coding assistants (GitHub Copilot, Amazon Bedrock agents), data analysis (Google Vertex AI pipelines, Azure AI Foundry), and autonomous task execution (CrewAI, AutoGPT derivatives).
Each of these carries distinct security requirements based on the tools available, the data accessed, and the consequences of a compromised action.
The AI agent threat landscape in 2026
The threat taxonomy for AI agents is broader than most teams realise. Here are the attack vectors that matter in production.
Prompt injection (direct and indirect)
The #1 risk in the OWASP Top 10 for LLM Applications. Direct injection manipulates the agent through user-supplied input. Indirect injection hides malicious instructions in data sources the agent retrieves — emails, documents, web pages, database records. Indirect injection is harder to detect and more dangerous because the agent trusts the data it fetches.
Data exfiltration
Agents with tool access can be coerced into extracting sensitive data through their outputs or tool calls — sending confidential information to external endpoints, embedding data in API calls, or writing it to accessible locations. This is distinct from PII leakage: data exfiltration is intentional (by the attacker), PII leakage is incidental.
PII and sensitive data exposure
Agents processing customer data can leak personally identifiable information — SSNs, credit card numbers, API keys, tokens, health records — in their responses, logs, or tool call parameters. OWASP elevated Sensitive Information Disclosure to the #2 risk in 2025, up from #6.
Tool call abuse
Agents call tools. Attackers manipulate which tools get called, with what parameters, and in what sequence. An agent with database access can be prompted to run destructive queries. An agent with email access can be prompted to send phishing messages. No major security vendor currently scans tool calls — this is a genuine capability gap in the market.
Memory poisoning
Agents with persistent memory (RAG stores, conversation history, long-term knowledge bases) can have their memory corrupted. Poisoned data sources inject false beliefs about security policies, user permissions, or system state that persist across sessions and influence future decisions.
Privilege escalation
Agents operating with broad permissions can be tricked into actions beyond their intended scope — accessing admin functions, modifying system configurations, reading restricted data. This is the agent-specific version of a classic security vulnerability, amplified by the agent's ability to reason about how to achieve unauthorized goals.
Lateral movement
In multi-agent architectures, a compromised agent can use its access to communicate with or manipulate other agents in the system. If Agent A has tool access and Agent B trusts Agent A's outputs, compromising A gives the attacker a path to B's capabilities.
Supply chain attacks
Poisoned models, compromised dependencies, and malicious packages in the agent's toolchain. A backdoored model or a compromised LangChain plugin can introduce vulnerabilities before your agent processes a single user request.
Denial of Wallet (DoW)
Attackers triggering expensive LLM calls, tool executions, or API requests to inflate operational costs. Unlike denial-of-service, the goal isn't to crash the system — it's to make it prohibitively expensive to run.
Cascading failures
In multi-agent systems, a failure or compromise in one agent can cascade through the system. Without circuit breakers and trust boundaries between agents, a single injection can trigger a chain reaction across your entire agent infrastructure.
How agent security actually works
Most agent security content stops at “best practices” without explaining the detection mechanism. Here's how runtime agent security works at a technical level.
The three-layer detection model
Effective agent security requires multiple detection layers because no single method catches everything:
Layer 1 — Pattern matching (regex, <5ms): Fast, deterministic scanning for known attack patterns, PII formats (SSN, credit card, API key regex), and policy violations. Runs on the critical path. Catches ~60% of known injection patterns. Think of this as the first filter — fast and cheap, catches the obvious stuff.
Layer 2 — Vector similarity (~30ms): Semantic analysis using embedding models to detect attacks that don't match known patterns but are semantically similar to known threat categories. Catches novel phishing variants, obfuscated injections, and semantically disguised exfiltration attempts. Can run asynchronously for zero-impact deployments.
Layer 3 — LLM judge (<500ms async): A dedicated LLM evaluates suspicious inputs/outputs in context, understanding the full conversation and determining whether a request is legitimate or adversarial. Required for sophisticated, context-dependent attacks that evade pattern and vector layers.
All three layers are needed for comprehensive coverage. L1 alone misses novel attacks. L2 alone misses known patterns that are fast and cheap to catch. L3 alone is too slow and expensive for every request.
In-process vs. cloud-API architecture
Where the scanning happens matters as much as how it scans.
Cloud-API architecture sends your agent's data to an external service for scanning. The data leaves your infrastructure, adds network latency (50–200ms per call), and creates a dependency on a third-party service being available.
In-process architecture runs the scanner within your application process. The data never leaves your infrastructure. L1 scanning adds <5ms overhead — less than a DNS lookup. No external dependency on the critical path.
For teams with data residency requirements (GDPR, HIPAA, SOC2), in-process architecture isn't a nice-to-have — it's a compliance requirement. Your agent's data, your customers' PII, and your tool call parameters should not be transiting to a third-party cloud for security scanning.
What gets scanned
Runtime agent security scans three surfaces:
- Inputs — Everything entering the agent: user messages, retrieved documents, API responses, file contents. This is where injection attacks originate.
- Outputs — Everything the agent produces: responses to users, data written to storage, content passed to downstream systems. This is where data exfiltration and PII leakage manifest.
- Tool calls — Every function call the agent makes: the tool name, parameters, and execution context. This is where privilege escalation and unauthorized actions happen. Most security tools don't see this layer at all.
See what your agents are doing in production.
Scan your first agent session in under 2 minutes.
Try the Free Scanner8 best practices for securing AI agents
This framework covers the core security controls every production agent should have, whether you build them yourself or use a platform.
1. Input validation and sanitisation
Every input to the agent must be validated before processing. This includes user messages, retrieved documents, API responses, and tool outputs. Implement pattern-based detection for known injection signatures, semantic analysis for novel attacks, and content-type validation for structured inputs. Reject or flag inputs that match threat patterns rather than relying on the LLM to “ignore” malicious instructions.
2. Output validation and guardrails
Scan all agent outputs before they reach users or downstream systems. Check for PII leakage (SSN patterns, credit card numbers, API keys), data exfiltration attempts (encoded data, suspicious URLs in responses), and policy violations (off-topic responses, unauthorized disclosures). Output guardrails are your last line of defense when an injection bypasses input filters.
3. Least privilege for tool access
Agents should have access only to the tools and data they need for their specific task. Use a declarative policy engine to define which tools each agent can call, with what parameters, and under what conditions. Deny by default. A customer support agent does not need database write access. A coding agent does not need email access.
4. Human-in-the-loop controls
For high-risk actions — financial transactions, data deletion, external communications, permission changes — require human approval before execution. Implement blocking policies that halt the agent and alert a human when a sensitive tool call is detected. This isn't about slowing agents down; it's about creating checkpoints for actions with irreversible consequences.
5. Memory and context security
Validate data entering the agent's long-term memory and RAG stores. Implement integrity checks on retrieved context. Monitor for memory poisoning attempts that inject false information into knowledge bases. Treat the agent's memory as a writable attack surface, not a trusted data store.
6. Monitoring and observability
Log every agent interaction: inputs, outputs, tool calls, decisions, and policy evaluations. Build session-level visibility so you can trace an attack from initial injection through tool calls to final output. Set up real-time alerting for anomalous behaviour — unusual tool call patterns, spikes in flagged inputs, unexpected data access. You can't secure what you can't see.
7. Identity and access management for agents
Agents are non-human identities that authenticate to APIs, access data stores, and interact with external services. Manage agent credentials with the same rigour as human credentials: short-lived tokens, workload identity federation where supported (OIDC, SAML 2.0), secure credential storage (HSMs for high-security environments), and regular credential rotation. Never hardcode API keys in agent code.
8. Incident response planning
Define procedures for when an agent is compromised: isolate the agent, revoke its credentials, preserve forensic logs, assess blast radius, and notify affected parties. Your incident response plan should account for agent-specific scenarios — a compromised agent with tool access requires faster containment than a compromised chatbot with no action capabilities.
The AI agent security market in 2026: why independence matters
The agent security market has consolidated rapidly. Understanding who owns what matters when you're choosing a tool that sits between your agent and your users.
- Lakera — Acquired by Check Point for ~$300M (September 2025). Now part of Check Point's enterprise security platform. The standalone product is being absorbed.
- Protect AI — Acquired by Palo Alto Networks for ~$500M+ (completed July 2025). Integrated into Palo Alto's Cortex platform. Enterprise procurement required.
- Promptfoo — Acquired by OpenAI (March 2026). Now part of OpenAI's safety infrastructure. Independent development discontinued.
What this means for developers: every major standalone agent security tool has been absorbed by an enterprise vendor. The self-serve, developer-first options are being replaced by enterprise platforms that require procurement cycles, sales calls, and annual contracts.
Rune is the last independent platform in the category. Not because independence is a marketing angle — because it determines how the product evolves. Independent means: self-serve signup, free tier that stays free, no sales calls for non-enterprise tiers, roadmap driven by developer needs rather than an acquirer's enterprise GTM strategy.
When evaluating tools, ask: who owns this, and whose priorities drive the roadmap?
Compliance frameworks for AI agent security
Regulatory pressure on AI systems is intensifying. Here are the frameworks that apply to agents in production:
OWASP Top 10 for LLM Applications (2025) — The industry standard risk taxonomy. Prompt injection remains #1. Sensitive Information Disclosure jumped to #2. New categories added for vector/embedding weaknesses and system prompt leakage. Excessive Agency (LLM06) now explicitly addresses agentic architectures.
EU AI Act — Becomes fully applicable 2 August 2026. High-risk AI systems must demonstrate accuracy, robustness, and cybersecurity protection. Requires risk management systems, technical documentation, and serious incident reporting. If your agents operate in regulated environments in the EU, runtime security scanning and audit trails are becoming compliance necessities.
NIST AI Risk Management Framework (AI RMF) — Provides a structured approach to AI risk: govern, map, measure, manage. Maps to agent security through risk assessment methodologies and continuous monitoring requirements.
ISO 42001 — The international standard for AI management systems. Defines requirements for establishing, implementing, and maintaining an AI management system.
MITRE ATLAS — Adversarial Threat Landscape for Artificial-Intelligence Systems. Provides a knowledge base of adversary tactics and techniques specific to AI, useful for threat modelling agent architectures.
Google SAIF (Secure AI Framework) and CoSAI (Consortium for Secure AI) — Industry-led initiatives establishing shared security standards for AI systems. Worth tracking as they mature.
For teams running agents in healthcare (HIPAA), financial services (SOC2, PCI DSS), or handling EU citizen data (GDPR), runtime security with audit trails isn't optional infrastructure — it's a compliance requirement with specific deadlines.
Securing multi-agent architectures
Multi-agent systems — where multiple agents collaborate, delegate tasks, and share data — introduce security challenges that don't exist in single-agent deployments.
Trust boundaries: Not all agents in a system should be trusted equally. OWASP defines four trust levels: untrusted (external-facing), internal (organisation-controlled), privileged (elevated access), and system (infrastructure-level). Each trust level requires different validation rules and scanning depth.
Inter-agent communication: Messages between agents are attack vectors. A compromised agent can send malicious instructions to other agents that trust its output. Implement message validation, cryptographic signing for high-security environments, and replay attack prevention for inter-agent protocols.
Cascading failures: Without circuit breakers, a single compromised agent can cascade through the system. If Agent A calls Agent B with a poisoned instruction, and Agent B calls Agent C, the blast radius expands with each hop. Runtime scanning at every inter-agent boundary prevents cascade propagation.
Per-session scanning applied at each agent boundary is the most effective pattern: scan every input and output at each trust boundary, not just at the system perimeter.
Adding agent security in 3 lines of Python
Here's what integration looks like with the frameworks developers actually use.
LangChain
from runesec import Rune
from langchain.agents import AgentExecutor
rune = Rune()
# Scan user input before the agent processes it
scan_result = rune.scan(user_input)
if scan_result.is_threat:
return scan_result.blocked_response
# Run your agent as normal
response = agent_executor.invoke({"input": user_input})OpenAI SDK
from runesec import Rune
from openai import OpenAI
rune = Rune()
client = OpenAI()
# Scan before sending to the model
scan_result = rune.scan(user_message)
if scan_result.is_threat:
return scan_result.blocked_response
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": user_message}]
)
# Scan the output before returning to the user
output_scan = rune.scan_output(response.choices[0].message.content)Anthropic SDK
from runesec import Rune
import anthropic
rune = Rune()
client = anthropic.Anthropic()
scan_result = rune.scan(user_message)
if scan_result.is_threat:
return scan_result.blocked_response
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[{"role": "user", "content": user_message}]
)
# Scan the output before returning to the user
output_scan = rune.scan_output(response.content[0].text)MCP (Model Context Protocol)
MCP introduces a specific security consideration: it standardises how agents connect to external tools and data sources, which means it also standardises the attack surface. A malicious MCP server can serve poisoned context. A compromised tool endpoint can execute arbitrary actions. Scanning MCP tool calls — not just LLM inputs and outputs — is critical for any MCP-based agent architecture.
from runesec import Rune
rune = Rune()
# Scan MCP tool calls before execution
tool_scan = rune.scan_tool_call(
tool_name="database_query",
parameters={"query": tool_params["query"]}
)
if tool_scan.is_threat:
return tool_scan.blocked_responseYour agents are already in production. Now see what's hitting them.
10K events/month. No credit card. No sales call. pip install runesec and go.
Start Scanning FreeWhat to look for in an agent security tool
If you're evaluating tools (or building your own), here are the criteria that matter from a developer's perspective:
Scanning depth: Does it scan inputs only, or inputs + outputs + tool calls? Most tools only scan inputs. Tool calls are where the most dangerous actions happen.
Detection layers: Pattern matching alone misses novel attacks. Vector similarity alone is slow for known patterns. LLM-based judgment alone is expensive. You need multiple layers with clear trade-offs between speed, cost, and coverage.
Latency impact: What overhead does it add to your agent's critical path? Cloud-API scanning adds 50–200ms per call. In-process L1 scanning should add <5ms. Async layers should add zero critical-path latency.
Integration effort: How many lines of code? Does it require architectural changes? Can you add it to an existing agent without refactoring? If it takes more than 10 minutes to integrate, adoption will stall.
Data residency: Does your agent's data leave your infrastructure for scanning? For teams with GDPR, HIPAA, or SOC2 requirements, this is a hard constraint, not a preference.
Vendor independence: Who owns the tool, and whose priorities drive the roadmap? After the acquisition wave, this question matters more than it did a year ago.
False positive rate: Security scanning that blocks legitimate requests kills user experience. Evaluate against real agent traffic, not synthetic benchmarks. Rune reports 0 false positives on the OWASP top-10 LLM attack set — but always test against your own traffic patterns.
Policy flexibility: Can you define custom rules? Can you set different policies for different agents? Can you whitelist specific patterns for your use case? Rigid, one-size-fits-all scanning creates friction.
Measuring agent security effectiveness
Security without measurement is guesswork. Here are the metrics that matter for agent runtime protection:
Detection rate: What percentage of real attacks does the system catch? Benchmark against known attack sets (OWASP top-10 for LLMs) and novel attacks in production. Rune's three-layer pipeline catches progressively: L1 ~60% of known patterns, L1+L2 catches semantic variants, L1+L2+L3 provides comprehensive contextual coverage.
False positive rate: How often does the scanner block legitimate requests? Even 1% false positives on a high-traffic agent means hundreds of frustrated users per day. Aim for zero on standard traffic patterns.
Mean time to detect (MTTD): How quickly does the system identify an active attack? In-process L1 detection operates in <5ms — the attack is flagged before the agent can act on it. Async layers (L2, L3) provide deeper analysis within seconds.
Mean time to respond (MTTR): How quickly does the system block or contain a detected threat? Blocking policies should execute in the same request cycle — not after the agent has already acted on the malicious input.
Coverage: What percentage of your agent fleet is monitored? Shadow agents — agents deployed without security scanning — are the blind spot. Track deployment coverage as a metric.
Where agent security matters most
Customer support agents — Processing customer data, accessing CRM systems, handling billing information. Attack vector: indirect injection via customer-submitted tickets containing malicious instructions. Risk: PII leakage, unauthorized account changes.
Coding agents — Executing code, accessing repositories, modifying infrastructure. Attack vector: malicious code suggestions that introduce vulnerabilities or exfiltrate secrets. Risk: supply chain compromise, credential theft.
Internal workflow agents — Connecting to databases, file systems, email, and enterprise tools. Attack vector: prompt injection through documents the agent processes. Risk: lateral movement across internal systems, data exfiltration.
Data analysis agents — Querying databases, processing sensitive datasets, generating reports. Attack vector: injection through data sources, tool call manipulation to extract unauthorized data. Risk: regulatory violation, competitive intelligence leakage.
Each use case requires different policy configurations, different tool call restrictions, and different alerting thresholds. One-size-fits-all scanning doesn't work.
Frequently asked questions
What are the risks of AI agents?
AI agents face risks that don't exist in traditional applications: prompt injection (direct and indirect), data exfiltration through tool calls, PII leakage in outputs, memory poisoning, privilege escalation, lateral movement in multi-agent systems, and supply chain attacks through compromised models or dependencies. Rune's production data shows a 14.2% injection rate — roughly 1 in 7 agent sessions contains an attack attempt.
How do you secure AI agents?
Securing AI agents requires runtime protection at the I/O layer: scanning every input, output, and tool call for threats in real time. Core controls include input validation, output guardrails, least-privilege tool access, human-in-the-loop for high-risk actions, memory integrity validation, monitoring and observability, agent identity management, and incident response procedures. The most effective approach combines multiple detection layers (pattern matching, vector similarity, LLM-based judgment) to catch both known and novel attacks.
What is agentic AI security?
Agentic AI security refers to protecting AI systems that take autonomous actions — calling tools, accessing data, making decisions — as opposed to passive AI systems that only generate text. The key difference is the expanded attack surface: agentic systems can be manipulated into taking real-world actions with consequences, not just producing incorrect responses.
What is the OWASP Top 10 for AI agents?
The OWASP Top 10 for LLM Applications (2025 edition) lists the ten most critical security risks for LLM-based systems, including agents. The top risks are: Prompt Injection (#1), Sensitive Information Disclosure (#2), Supply Chain Vulnerabilities, Data and Model Poisoning, Improper Output Handling, Excessive Agency, System Prompt Leakage, Vector and Embedding Weaknesses, Misinformation, and Unbounded Consumption. OWASP also published a dedicated AI Agent Security Cheat Sheet covering multi-agent architectures, trust levels, and implementation-level best practices.
What is the difference between AI security and AI agent security?
AI security is the broad field covering all AI system threats: model attacks (adversarial examples, model extraction), data attacks (poisoning, inference), and deployment attacks (supply chain, infrastructure). AI agent security is a subset focused specifically on autonomous AI systems that take actions — the additional risks created by tool access, persistent memory, and multi-step decision-making. Agent security requires runtime protection at the I/O and tool-call layer, which is beyond the scope of traditional AI security approaches focused on model-level or training-level threats.
What is Agent Detection and Response (ADR)?
Agent Detection and Response (ADR) is a runtime security category for AI agents focused on detecting and blocking active threats across every input, output, and tool call in real time. It is the agent-specific equivalent of Endpoint Detection and Response (EDR) for traditional endpoints. ADR sits on the critical path between the agent and users, tools, and data sources, scanning each interaction for prompt injection, data exfiltration, PII leakage, and policy violations. ADR differs from AI Security Posture Management (AI-SPM), which focuses on governance, inventory, and compliance posture rather than real-time threat blocking.
What is AI Security Posture Management (AI-SPM)?
AI Security Posture Management (AI-SPM) is a governance category for AI agents focused on discovering and governing agents across an organisation: inventory, policy definition, compliance posture, and risk assessment. AI-SPM tools answer questions like 'how many agents do we run', 'what data can they access', and 'which policies apply'. They are complementary to Agent Detection and Response (ADR), which handles runtime threat detection. AI-SPM is governance; ADR is detection and response. Most mature agent security programmes use both.
What is indirect prompt injection and how is it different from direct injection?
Direct prompt injection is an attacker typing malicious instructions into the agent through user input. Indirect prompt injection hides malicious instructions in data sources the agent retrieves on its own — emails, documents, web pages, database records, RAG stores. Indirect injection is harder to detect because the agent trusts its own data pipeline and treats the poisoned content as legitimate context. It is also more dangerous for autonomous agents because the attack can fire on general-sounding queries (e.g. 'summarize my inbox') without any explicit attacker interaction.
Does the EU AI Act apply to AI agents?
Yes. The EU AI Act becomes fully applicable on 2 August 2026. High-risk AI systems — which includes many agentic deployments in regulated sectors — must demonstrate accuracy, robustness, and cybersecurity protection. The Act requires risk management systems, technical documentation, post-market monitoring, and serious incident reporting. For AI agents in the EU, this means runtime security scanning and audit trails are becoming compliance requirements with specific deadlines, not optional infrastructure.
How much does AI agent security cost?
AI agent security pricing varies by model. Developer-first ADR platforms like Rune offer a free tier (10,000 events per month, no credit card, no sales call) with paid plans from $49/month for small teams to $499/month for growth-stage deployments, plus enterprise pricing for larger fleets. Enterprise AI security platforms from Check Point, Palo Alto Networks, and other acquirers typically require annual contracts and procurement cycles with significantly higher minimum commitments.
Continue reading
Your agents are in production. Are they protected?
Rune scans every input, output, and tool call your AI agents process. Three-layer detection. In-process architecture. <5ms on the critical path. 52 detection patterns. 0 false positives on OWASP top-10.
The free tier gives you 10,000 events per month. No credit card. No sales call. pip install runesec, add 3 lines, and see what's hitting your agents.