Gemini Enterprise agent evals go GA

Google announced on July 31 that evaluation tooling inside its Gemini Enterprise Agent Platform has reached general availability, giving developers a standing way to measure and compare agents and models across both development and production. The tooling spans three pieces: over 20 pre-built metrics covering quality, safety, and grounding, plus “adaptive rubrics” that adjust judging criteria per test case, and support for custom code-based or LLM-as-judge metrics stored in a shared registry.

The experiment side lets teams run evaluations locally or server-side with results stored in Cloud Storage, generate test cases automatically, simulate multi-turn users, and run an environment simulator that tests how an agent handles a backend that’s failing. On the monitoring side, the platform now runs continuous evaluation against live production traffic, tracking scores over time and alerting on drift, without a team having to build a custom data pipeline to do it. The tooling is accessible through the Agent Platform SDK, its command-line interface, and a web UI.

What it means for operators

Agent evaluation has mostly been a build-it-yourself problem: teams stitch together ad hoc test sets, manual review, and whatever logging happens to exist. Google’s GA release is worth treating as a checklist even if you don’t use the platform itself: pre-built quality and safety metrics, a way to simulate adversarial or multi-turn users before shipping, and continuous drift monitoring against real production traffic once you have. If you’re already on Google’s agent stack, this removes a real reason to keep a custom eval pipeline running in parallel. If you’re not, it’s a useful bar to compare against your own verification habits and how you scope and monitor long-running agent jobs, particularly the production drift monitoring piece, which is the part most homegrown eval setups skip entirely.