Quality & operations
AI Agent Security
Agents introduce attack surfaces a normal chatbot never has to think about.
Giving an AI system the ability to call tools and take action also gives it a new attack surface: an agent that can be tricked into calling the wrong tool, leaking sensitive data, or acting on instructions hidden inside content it reads is a genuinely different security problem than a normal chatbot answering questions.
AI agent security borrows from traditional application security, but adds new risks specific to language models reasoning over untrusted content and holding real tool permissions.
Key takeaways
Prompt injection: the defining agent risk
Prompt injection happens when content an agent reads — a web page, an email, a document — contains instructions designed to hijack the agent’s behavior, such as telling it to ignore its original task and instead exfiltrate data or call a dangerous tool.
This risk barely exists for a simple chatbot answering isolated questions, but becomes serious the moment an agent reads content it did not fully control and can act on tools as a result.
Scoping tool permissions tightly
An agent should only have access to the specific tools and permission scopes its current task actually requires, not broad, standing access "just in case." A research agent does not need permission to send emails; a scheduling agent does not need database delete access.
Least-privilege tool scoping limits the damage even when an agent is successfully manipulated into attempting a harmful action, since the underlying permission simply is not there.
Data leakage and untrusted content
Sensitive information can leak through more channels than expected: a model repeating confidential context in its output, verbose logs capturing data that should not be retained, or a tool result containing data that gets passed along further than intended.
Treat every piece of content an agent reads from outside your direct control — web pages, documents, user-submitted files — as untrusted input, the same discipline applied to user input in traditional web security.
Put this into practice