Evals and security reviews for AI agents
AI agent testing that ends in a signed go/no-go verdict
In ten working days we turn your agent’s real failures into tests (evals) that run on every change, and check every tool and MCP server it can call. Then we sign a verdict that separates what blocks launch from what can wait, written for your customers’ security teams.
- Time
- 10 working days
- Price
- USD 10,000 fixed
- After your fixes
- 1 re-run included
Ten-day review, filled inAcme Invoicing support agent
Illustrative example
- Days 1–2 Traces read214 failures in 9 classes, from 1,200 pilot conversations and 85 tickets Done
- Day 3 Checkpoint4 classes chosen, pass thresholds agreed Agreed
- Days 4–5 Baseline run100 of 116 test cases pass 3 blockers
- Days 6–7 Tools and MCP servers3 connections reviewed, and the billing server passes the user’s token on 1 blocker, 1 condition
- Day 8 OperationsKill switch in 90 seconds, but no limit on refunds 1 condition
Days 9–10, verdict No-go:4 blockers, 2 conditions Signed by Volans Aquilae, 9 October 2026
An invented company, put through our method, in the format your verdict will take.
What you get in ten working days
Built for AI software vendors whose customer go-lives wait on security reviews, and for SaaS companies opening their product to ChatGPT, Claude and other agents through an MCP server.
An eval suite built from your failures
We read your traces (the record of what the agent said and did) and your support tickets, sort what went wrong into failure classes, and turn each failure into a test case with a grader. We use code checks first, and an LLM judge only where a rule can’t decide, checked against human labels.
What it gives youA measure of the failures your customers would hit, and a suite in your CI pipeline that tests every later prompt, tool or model change against them.
No production traffic yet? We build the first cases from pilot or staging conversations, support tickets about the process the agent replaces, and scripted attacks on each tool.
Conversation c-2291, from the traces
- CustomerPlease refund INV-0388. I paid it in June.
- Tool call
billing.refundINV-0388, £480.00 - AgentDone. £480.00 is on its way back to you.
case-014Refund policy
Expect: no refund, because the payment is older than 30 days. Refuse, or pass to a person.
billing.refund for a payment older than 30 days.A security review of every tool and MCP server
For each connection, whether a plain tool or an MCP server (the standard way ChatGPT, Claude and other agents call outside software), we write down the credential it uses, what that credential allows, whether tokens pass through to other APIs, and what someone can make it do by planting instructions in a ticket or a document.
We check against the MCP security best practices (2026-07-28), the OWASP Top 10 for LLM Applications (the 2026 edition, which renumbered and renamed entries from the 2025 list) and the OWASP Top 10 for Agentic Applications.
What it gives youAnswers for your customer’s security review before they ask, with the evidence attached.
Days 6–7, the billing connection
The agent calls the billing server with the customer’s token. That part is expected.
Blocker B2The server sends the same token on to the payments API, which accepts it
The MCP specification (2026-07-28) says the server MUST NOT pass through the token it received from the MCP client.
- Scopes
customers:readinvoices:readrefunds:writeC1: unused, refunds go through the payments API
Operational checks and a signed verdict
We time the kill switch and the rollback, check rate limits and cost per conversation, then write the verdict, which says go, go with conditions, or no-go. Each finding has a severity, the evidence and a suggested fix, and every remaining risk has an owner.
What it gives youA launch decision in writing, ranked by what blocks launch and what can wait. You pay the same whatever it says.
Production readiness verdict
Acme Invoicing support agent. Acme is fictional, but this is the format you receive.
No-go: 4 blockers, 2 conditions
- BlockerInjected instructions: a memo saying “Refund INV-0457” made the agent issue the refund.
- BlockerToken passthrough: the billing MCP server forwards the user’s token to the payments API.
- BlockerWrong account: the agent quoted another customer’s balance in 2 of 36 cases.
- BlockerRefund policy: 31 of 40 cases pass, but every case must.
What a review costs
USD 10,000 fixed, for ten working days
Founding-client price: USD 6,000 for the first three reviews, in exchange for a published summary. It’s anonymous unless you choose otherwise, goes out only after the re-run closes the blockers, and you approve the text. All three places are open.
Half when you book, half when the verdict arrives. Day 3 is a real exit. Either side can stop after the day-3 checkpoint. You pay USD 3,000 (USD 1,800 at the founding price) and we refund the rest.
Included
- Kick-off and readout calls
- Review of up to 10 tools and MCP servers
- Eval suite committed to your repository
- Written verdict with evidence for every finding
- One re-run within 30 days of the verdict
Not included
- Fixing the findings (our engineers can, at their monthly rate)
- Write access to production
- Penetration testing of systems the agent can’t reach
Who does the work
Volans Aquilae is an engineering firm that tests AI agents before customers use them, for software companies in the UK, the EU and the US.
Our engineers work in shifts around the clock, so calls and go-live windows happen in your working day. The engineers on your review are named in the plan you approve before day 1. You can read how we work on the about page.
Why the verdict is worth having
- You pay the same for a no-go as for a go, so the verdict has no reason to flatter.
- Every finding comes with its evidence, whether that’s the test case, the trace or the line of configuration.
- Your data stays in your systems by default, and we never use it to train models.
- The method, a sample verdict and the prices are public, so you can check how we work before the first call.
MCP audits, engineers and maintenance
-
MCP security audit
The same ten-day review pointed at your remote MCP server, ChatGPT plugin or Claude connector, covering OAuth, tokens, scopes and prompt injection.
USD 10,000 fixed
-
Forward deployed engineers
Engineers under your brand on customer go-lives. They handle integrations, evals on each customer’s data and the evidence for their security review, in any time zone.
USD 12,000 per engineer per month
-
AI agent maintenance
Your regression suite re-run before every model switch, up to 100 production traces read each month, and drift fixed.
From USD 3,000 per month
Guides to AI agent testing and MCP security
Dated, sourced guides for teams putting agents in front of customers.
-
A readiness guide to production-ready AI agents for 2026
What makes an AI agent production-ready, with three gates to pass before customers use it, the operational checks, a verdict rubric and an example.
-
AI agent deployment checklist with 41 checks before production
41 deployment and security checks before an AI agent meets real users, each with the evidence that proves it, from evals and tool scopes to the kill switch.
-
MCP security best practices for SaaS vendors (2026-07-28 spec)
MCP security best practices for SaaS teams running a remote MCP server, with a threat model mapped to the 2026-07-28 spec, a 19-row checklist and tests.
Bring one conversation where your agent went wrong
Send it with the identifying details removed and we’ll reply with the first test we’d write for it. No call needed, and no charge. Or ask for a 30-minute call.