Чем дольше занимаюсь безопасность ИИ-агентов в разработке, тем лучше вижу тенденцию: индустрия обсуждает, что агенты умеют делать, но реже говорит о том, какие решения и как проще внедрять, чтобы сделать агентскую инфраструктуру безопаснее. Вместе с моими коллегами Ратмиром Самархановым и Андреем Погирейчиком мы решили проверить гипотезу: «наши ИИ‑агенты в разработке могут быть скомпрометированы и существуют простые средства для контроля их безопасности». Расскажем о первых результатах.
Announcing AI Agents Security Week and calling everyone back to school!
At Yandex School of Data Analysis, we just launched a week-long intensive: “AI Agents Security Week.”
We’ll be learning to save this brave new world of AI agents.
Our recent Saint Highload talk was the culmination of the AI-agent security research we ran while digging into secure agentic development. Audience feedback pushed it to the top of the ranking, a signal the findings land with engineers already wiring agents into their pipelines.
Takeaways and key focus on ADLC security now:
- Observability is the baseline. Also, we observed events on our own honeypot, a fake MCP server left open. Looked at stats, saw attackers probing AI-infra components. Looks like cyber deception gets a second life in agentic world.
- AI red teaming comes next, and it has to cover every component. The whole variety of attack prompts reduces to a starter corpus of queries, then let tooling escalate and adapt them to context. Continuously.
- MCP server is the agent’s hands that define how it changes external systems. That makes monitoring tool calls a priority of its own. And when agent doesn’t know what it’s doing, slap its hands and revoke its access.
Slides and tools are in the repo. Should probably teach the agent to convert slide decks into skills.