
Evaluating AI Agents: Accuracy Is the Wrong Metric
Agents have no single correct path, succeed partially, and behave differently run to run. What to measure instead of accuracy, and why failure shape matters most.

Agents have no single correct path, succeed partially, and behave differently run to run. What to measure instead of accuracy, and why failure shape matters most.

Multi-GPU scaling disappoints because the interconnect, not the silicon, is usually the limit. When more cards help, and when one bigger card is the answer.

A working rule from a PhD student: use AI where being wrong is cheap and detectable, avoid it where being wrong is expensive or invisible.

You cannot prevent an agent from making a bad decision. You can make sure a bad decision cannot do much. A practical security model built around tools, not prompts.

You cannot migrate cryptography you cannot find. How to build an inventory that covers the appliances, firmware and vendor products where most of it actually lives.

Combining an elliptic curve exchange with a post-quantum one is secure unless both break. It is widely supported already, and often just a setting at your CDN.

OpenAI’s published tables put Terra and Luna within a tenth of a point on professional agentic work and 48 points apart on long-context recall. Route by failure mode, not difficulty.

SQL injection was solved by separating code from data. Language models cannot make that separation, which is why prompt injection defences have to be architectural.

The useful distinction is not how many model calls a system makes, but whether it decides its own control flow. That difference changes testing, failure modes and security.

Memory capacity decides whether you can run a model at all, bandwidth decides how it feels, and compute is usually third. A practical sizing guide.