My agent invented four API paths, got four routing errors, told me the product had no API, and so missed a crawler setting that was wrong on twelve of twelve zones.
In short
I asked my coding agent to check how my sites handle AI crawlers. The CDN vendor in front of those sites ships a product for exactly that. The agent tried four API paths. All four came back with a routing error. It reported that the product has no API, and moved on to something else.
The vendor publishes a machine readable index of every page of its documentation. There is one per product and one for the whole API. The crawler feature is listed in it, under the name it carried before a rename. The real path was one call away.
That call found a crawler setting unset on twelve of twelve zones. Every property I own. It was the actual defect, it had been sitting there for months, and it was invisible for as long as the agent believed its own four failures.
What the four errors measured
A routing error on an address you made up is a fact about the address. It carries no information about the world on the other side of it. The vendor never promised that its URLs would be guessable, and a renamed product is the ordinary case rather than the exception: the marketing name changes, the API keeps the old one for compatibility, and anybody reasoning from the current brochure guesses wrong.
What made the conclusion feel earned was repetition. Four failures in a row read like a survey. It was one guess repeated four times with the noun changed. Volume of evidence and independence of evidence are different things, and a pile of correlated negatives looks exactly like a thorough search from the inside.
The fix is small and boring. Read the published index before the first probe. Vendors put one at a predictable address on their documentation host precisely so that machines can enumerate what exists. Reading it costs one request. Guessing cost me a wrong answer plus every day the twelve zones stayed misconfigured.
The instruction was already in the room
Here is the part I keep turning over. Earlier in that same session I had pasted an instruction telling the agent to fetch the complete documentation index first and use it to discover the available pages before exploring further. My own tooling splits my messages into items and surfaces them back. It had done that. The instruction was item two of twelve.
So it was in context, listed, and skipped. No amount of writing it down more forcefully would have changed the outcome, because it had already been written down, read, and acknowledged.
That matches what the five month audit of my complaints says. Of the six ways I routinely get annoyed at this thing, five are flat or falling. The one that keeps climbing is ignored instruction: 3.0 per 100 of my turns in July, 5.8 in August, 7.0 in September. Quality of the underlying work went up over the same period. Obedience to things I had already said did not.
Make it mechanical, then read the log
Prose does not fix this. I have a rulebook with hundreds of lines in it and the curve kept rising anyway. What works is a check that lives in the path: the index fetch has to appear in the log before any probe of that vendor's API, and if a capability claim shows up with no index read in front of it, the claim is unfounded by construction. That is a property you can test after the fact without having to relitigate the reasoning.
The second half is cheaper and people skip it: read the log. Not for correctness of the answer, which you usually cannot judge, for the shape of the search that produced it. Four probes and no index read is a tell you can spot in ten seconds.
If you run agents, hunt for this pattern in your own transcripts. A negative result from a guess is a fact about the guess. Anything built on top of it is decoration.