a green diff lied to me last week

a green diff lied to me last week. the commit landed, tests passed, and i moved on with the kind of confidence that comes from seeing a clean result in the repo. meanwhile, the system was quietly broken in prod the whole time. i found out because a user found it first, which is the worst way to learn that something you shipped has a hole in it. that kind of discovery sticks with you. it leaves a mark on how you judge your own work, because the pleasant feeling of a passing test can turn out to be a very small part of the real story.

the part that stings is how ordinary the false signal looked. nothing dramatic happened in the moment of the commit. there was no obvious alarm, no loud failure, no red flag in the place i was already checking. the diff looked green, the tests looked green, and that was enough for me to move on. that is the trap. a repo can look healthy while the live system carries the actual problem. when i build software solo with AI agents, i am often moving fast and trusting the shape of the work that sits in front of me. this was a sharp reminder that the shape of the work and the behavior of the live thing can part ways.

the receipt was simple and painful. the commit landed. the tests passed. the product still broke in production. a user found the issue before i did. that sequence matters because it shows exactly where the blind spot lived. i had evidence that my code fit the repo’s expectations, but i had no evidence that the live thing behaved the way i wanted it to behave. the number that matters here is the gap between those two scoreboards. one says the change is accepted by the repo. the other says the real world is actually handling it. i had been spending my attention on the first one and treating it like the second one.

the mechanism is easy to miss when you are in the middle of shipping. tests tell you about the code path you covered. prod tells you about the thing users actually touch. those are related, and they are also separate. a passing test can mean the code compiled, the expected path worked, and the repo-level checks were satisfied. live behavior asks a different question. it asks whether the thing still works after the commit leaves your hands and meets the rest of the system. once i saw that clearly, the mistake felt less mysterious. the failure was never that the tests passed. the failure was that i let passing tests stand in for proof that the live thing worked.

that led to a rule i actually trust now: a change is done when i have watched the live thing actually work. the commit is a checkpoint, and the tests are a checkpoint, and neither one earns the final stamp on its own. i want the moment where the thing is real outside the repo. i want to see it behave where users will feel it. that is the point where i stop telling myself the job is finished. for my own workflow, especially with AI agents in the loop, that rule creates a hard boundary around optimism. it keeps me from confusing a clean diff with a complete result.

if you build software, you can use the same lesson without changing your whole process. keep your tests. keep your commit discipline. then add one more standard before you call something shipped: watch the live thing work. if you cannot do that immediately, treat the work as still open. that one habit changes how you read your own progress. it pushes you toward reality instead of paperwork. it also gives you a better answer when a change looks good in the repo and still behaves badly in the product. that gap is where the useful learning lives, and it is where the honest definition of shipped has to sit.