Guides to AI agent readiness, MCP security and model updates
For teams putting AI agents in front of customers. Every guide is dated, names the sources it used, says what we think and what it can’t tell you.
Production readiness and evals
-
A readiness guide to production-ready AI agents for 2026
What makes an AI agent production-ready, with three gates to pass before customers use it, the operational checks, a verdict rubric and an example.
-
AI agent deployment checklist with 41 checks before production
41 deployment and security checks before an AI agent meets real users, each with the evidence that proves it, from evals and tool scopes to the kill switch.
-
How to build a golden dataset from production failures
Turn production traces and support tickets into a golden dataset, with a failure taxonomy template, a JSONL case schema and a Python runner.
-
How to calibrate an LLM as a judge for agent evals
How to use an LLM as a judge in agent evals, with one pass/fail criterion per class, TPR and TNR on held-out labels and a corrected pass rate.
MCP servers and agent security
-
MCP vs API, and when you need an MCP server
MCP vs API explained. What an MCP server adds on top of a REST API, the same task built both ways, and when a SaaS company needs one.
-
MCP security best practices for SaaS vendors (2026-07-28 spec)
MCP security best practices for SaaS teams running a remote MCP server, with a threat model mapped to the 2026-07-28 spec, a 19-row checklist and tests.
-
MCP OAuth and CIMD authorization on the 2026-07-28 spec
How MCP OAuth works on the 2026-07-28 spec, covering discovery, Client ID Metadata Documents, PKCE, iss and scopes, with conformance tests.
-
How to test an MCP server, from MCP Inspector to CI
How to test a remote MCP server on the 2026-07-28 spec, with conformance, contract and auth tests, tool-use evals and CI steps we ran locally.
-
MCP error codes explained, with the fix for each
What each MCP error means and how to fix it, from -32000 connection closed and -32001 timeouts to a missing Mcp-Session-Id and session not found.
-
MCP stateless migration checklist for the 2026-07-28 spec
A 12-step checklist for moving a remote MCP server to the stateless 2026-07-28 spec, with a check per step, keeping 2025 clients working.
-
AI red teaming vs penetration testing vs a readiness review
What a penetration test, an AI red team and a production readiness review each find in an AI agent, what each misses, which to run first.
-
OWASP Top 10 for Agentic Applications 2026, with a test per risk
A pre-launch test for each risk in the OWASP Top 10 for Agentic Applications (ASI01 to ASI10), with passing evidence and worked eval cases.
-
20 prompt injection test cases for AI agents
20 prompt injection test cases for AI agents, direct and indirect, through emails, documents, web pages, tool results and MCP tool descriptions.
Model updates and deprecations
-
A runbook for LLM regression testing after model updates
When a provider ships or retires a model, pin the IDs, re-run the suite with repeat samples, compare each failure class, then roll out or back.
-
LLM model deprecation dates for Claude, OpenAI and Gemini
Claude, OpenAI and Gemini model retirement dates, checked 5 November 2026, with cloud platform differences and a 30, 14 and 7 day checklist.
For AI software vendors
-
What buyers ask in an AI vendor security questionnaire
What enterprise buyers ask AI vendors about agents in 2026, the frameworks behind the questions, and an answer bank with evidence to attach.
-
Forward deployed engineer cost in 2026, hired or outsourced
US and UK forward deployed engineer salary data for 2026, the fully loaded cost of a hire, and a model for hiring, contracting or renting.
Downloads
- AI agent deployment checklist
41 checks with the evidence for each and a sign-off block. Markdown, to paste into an issue or pull request.
- LLM model deprecation dates
One record per model ID and platform, each with its source and the date we checked it. JSON, CC BY 4.0.
- Sample go/no-go verdict
The six-section verdict for a fictional billing agent. PDF.
- LLM judge calibration script
Measures an LLM judge against human labels and reports TPR, TNR, kappa and an error-corrected pass rate. Python, standard library only.
- MCP server test kit
The project behind our guide to testing an MCP server, with tests, evals and a CI workflow whose steps we ran locally. Zip, MIT licence.
New guides
Subscribe to the RSS feed. When a specification or a model changes in a way that affects a guide, we update the guide and record the change at the bottom of it.
Have these checks run on your agent in ten working days
A production readiness review runs the gates in these guides on your agent for USD 10,000 and ends with a signed verdict.