tomerhakak/agentprobe
🔧 Tooltomerhakak
A pytest-inspired framework for recording, testing, and securing AI agents with built-in prompt injection fuzzing.
AgentProbe addresses the critical gap in AI agent reliability by providing a comprehensive testing suite tailored for LLM-based applications. The framework allows developers to treat agent behavior as testable code, facilitating the creation of reproducible test cases through interaction recording. Key technical features include a library of 35+ specialized assertions that validate agent outputs, logic, and tool usage.
Security is a primary focus, with integrated prompt injection fuzzing tools that help identify vulnerabilities before deployment. Beyond security, AgentProbe provides deep observability into agent operations, including real-time cost tracking to optimize token usage and latency. Built with a local-first philosophy, it integrates seamlessly into existing Python development workflows, making it an essential utility for teams moving from prototype to production. Whether you are building complex multi-step agents or simple RAG pipelines, AgentProbe provides the necessary infrastructure to ensure consistent performance and safety.
💡Highlights
- ├─35+ built-in validation assertions
- ├─Automated prompt injection fuzzing
- └─Real-time token cost tracking
🎯For
- ├─AI Engineers
- ├─LLM-Ops Specialists
- └─Security Researchers