New features, improvements, and fixes in Agenta.
We have rebuilt the playground from scratch to be the main workspace for your AI applications.
We have added two major features:
You can attach evaluators to the playground to score outputs as you edit.
You can load test sets. This allows you to work on your evals and add test cases directly from the playground.
Agenta now has a dark theme across the whole app. Switch between light, dark, and system from the top bar, next to the version label. Pick system and Agenta follows your operating system. Your choice is saved for next time.
The most useful thing you can do when building an LLM app is read your traces. You find the failures, label what went wrong, and turn the worst ones into test cases. That loop used to happen in spreadsheets. Annotation queues bring it into Agenta.
Build a queue from traces or test set rows, attach a scoring schema (ratings, dropdowns, rubrics, or free text), and route it to reviewers. When the queue is done, export it as a labeled test set. The annotations come along as columns, so the work feeds straight into your evaluators.
All application invocation endpoints (/generate, /generate_deployed, /test, /run) are now unified into a single POST /services/{service}/v0/invoke endpoint. The new format uses structured references for targeting applications and environments, and returns outputs under data.outputs with trace_id and span_id for observability.
The old endpoints remain available temporarily via an adapter. See the full guide for before/after examples.
You can now trigger automations when a prompt deployment happens in Agenta. Send the event to any HTTPS endpoint, or call GitHub directly with repository_dispatch or workflow_dispatch.
This makes it easier to connect prompt deployments to CI, repository sync jobs, and pull request workflows. If your GitHub workflow needs the latest prompt content, fetch it from Agenta during the run and commit the result back to your repo.
Learn more: Webhooks | GitHub Automations
You can now connect 150+ external tools to your prompts directly from the playground. Browse integrations like Gmail, Slack, Notion, Google Sheets, and GitHub. Authenticate with OAuth, attach tool actions to your prompt, and execute tool calls with one click. Use Google Sheets or Notion as data sources for RAG, send emails from your prompt, or automate developer workflows.
You can now refine prompts with AI directly in the playground. Click the wand icon on any prompt, describe what you want to improve in plain English, and get back a refined version with a summary of changes. Each refinement builds on the last, so you can iterate. Toggle diff view to see exactly what changed, edit the result before applying, or use the quick "Optimize using best practices" shortcut.
Agenta has new enterprise features. You can now create separate organizations for different teams or clients, each with its own billing, projects, and roles. We added SSO with any OIDC provider (Okta, Azure AD, Auth0, OneLogin, Google Workspace). You can enforce SSO-only for an org and disable password login. Domain verification lets you claim your company domain so new users with matching emails join automatically. We also launched a US region for customers who need their data to stay in the United States.
SSO, and domain verification are on Business and Enterprise plans. The US region is on all plans.
Prompts multiply fast when you're building agents or managing multiple use cases. Finding the right one becomes guesswork.
You can now create folders and subfolders to organize prompts. Drag prompts between folders, create nested hierarchies, and search across everything. Folder URLs are shareable.
New users now get an onboarding widget with guided walkthroughs. The widget appears in the sidebar and walks you through key features like the playground, evaluations, and observability. Each tour highlights UI elements as you go, so you learn by doing. Track your progress and revisit walkthroughs anytime.