Most agent demos work because someone hand-picked the prompt. Production agents fail differently: wrong tool chosen, hallucinated field values, sensitive data returned to the wrong user, quiet drift after a model swap. "How do I actually test this thing?" is the most under-answered question in agentic AI today, and the honest answer isn't one technique but four distinct layers, each catching failures the others miss. This session walks attendees through all four layers on Microsoft Foundry and the Microsoft Agent Framework, with working code at every step: Functional evaluation: judging both the output and the trajectory. Intent resolution, tool-call accuracy, task adherence, groundedness. When LLM-as-judge is good enough and when it quietly lies. Adversarial testing: running the AI Red Teaming Agent with PyRIT to probe for direct and indirect prompt injection (XPIA), prohibited actions, and data leakage. Reading Attack Success Rate without panicking. Continuous evaluation in CI/CD: wiring evals into a pull-request pipeline so a quality regression blocks a deploy instead of becoming a customer ticket. Multi-agent gotchas: what changes when agents call other agents over A2A or MCP, why deterministic replay is hard, and what to do about non-determinism in tests. Attendees leave with a public GitHub repo containing a sample agent, evaluation datasets, red-team configurations, and a CI pipeline they can drop into their own projects. Takeaways: A mental model, the agent test pyramid, for deciding what to test where. Concrete code for each layer, not slides about code. A clear-eyed view of what's still hard, especially in multi-agent workflows.
