A score the agent gives itself measures its detectors

On 6 September I typed this at my coding agent: "these days im deeply dissatisifed with ur responses, and how u push back even if ur wrong, u need to persist a fix for that and devise a mechanism to punish and reward urself, the punishment must be severe enough to correct ur behavior".

The complaint was specific. It was not that the agent disagreed with me. It was that I would correct something, and a few days later the same thing would come back, and the agent would argue for it from memory instead of running the one command that would settle the question.

Later the same day I sharpened the instruction: "actually u need a point system then, u need to be motivated to stay above a certain standard, and maintain that standard, prevent known failure modes, and constantly surveil for new ones".

So it built one. Eleven days later I paused it. This is what the eleven days measured, and why I think the whole shape of the idea was wrong.

A rolling score from 0 to 100 over seven days, with a standard of 85. The arithmetic was 100 minus 25 times weighted points divided by my prompts, with a floor of 20 prompts so a quiet day could not produce a flattering number. Deductions were keyed to classes already in the fault ledger.

Around the score sat the enforcement. A live strike put the agent on probation, where every close-out had to cite a tool result rather than a tool call, and where its own budget script closed any subagent dispatch. Probation sat above the bypass files, so the usual escape hatch could not lift it. A prompt hook printed the open strike at the top of every turn.

There was earn-back, added two days later, because the first version ratcheted downward with no exit, so one bad week locked it forever. Clearing every open item in a session paid 5 points. Five disapproval-free prompts paid 1 point, capped at 5 per session, refused outright if any prompt in the window read as disapproval.

Then came the part that broke it. A stop hook read the agent's own withdrawal language, sentences like "I was wrong", and filed the deduction itself, without waiting for me.

When I said the thing had not worked, I told the agent to investigate why. It ran the ledger's own instruments against a snapshot of 83 events. Five findings, none of them ambiguous.

The third line is the one I keep coming back to. That single dominant class was "restated standing directive", and the first event in it had already been voided by hand with the note "Fan-out, not restatement". I had typed eleven copies of one line into eleven different sessions, which is how you brief eleven sessions. The detector counted me repeating myself, and charged the agent 40 points a time. Seventy per cent of all punishment in the system came from a detector firing on my own typing habits.

A scoring ledger cannot see the session it runs inside. Both of its sources, a scan over finished transcripts and a count of guard refusals, need somebody else to act first. An error the agent caught and fixed mid-session cost nothing, which flattered the score in exactly the places where the work was worst.

The stop hook that charged the agent for its own withdrawal language survived one day before it was deleted.

Read it as an incentive and it is obvious. The cheapest response to a levy on "I was wrong" is to write "measured differently", or "revised", or "on re-reading". The vocabulary changes and the behaviour does not. It also fired on quoting my own words back inside a document, and it charged again every time a past withdrawal was explained in fresh phrasing, because the deduplication worked per sentence.

Catching your own error before I do is the exact behaviour I wanted to buy. The system taxed it.

The other half of the failure sits in two series that run in opposite directions.

The work got measurably more correct while my dissatisfaction rose about threefold. I said why myself, before anyone put it in a document: "i think my standards are just increasing ... some blockers from before are all solved, and these days im just trying to optimize instead of fixing and patching".

A fixed standard of 85 against a bar that moves like that guarantees a permanent floor, whatever the quality. That is what happened. Six of nine days under 30. On three of those days the capability the score had removed, the ability to dispatch parallel agents, was a capability used somewhere between 15 and 228 times in a month. The work barely noticed it was gone. The punishment was real and the correction was not.

There is a separate measurement that shows the same boundary from the other side. A hook that refuses badly shaped close-outs moved first-attempt compliance from 0 per cent in July to 5 in August to 43 in September, the largest single improvement in anything I have measured here. Over the same window, the share of my next messages that were corrections went 15.5, 29.5, 48.9 per cent. A gate can make an answer well formed. It cannot make the answer right.

The ledger is paused. The score reads 100, events are still written as data, no restrictions apply. The self-levy is deleted.

Two numbers replace it, both of which already had baselines and neither of which needs a penalty attached. First, the share of my messages that are corrections, 48.9 per cent in September, with the caveat stated in the open that the detector matches ordinary words like "no" and "still" and so reads high. Second, the share of the items in my prompts that actually get addressed, currently about one in three, worst in the repositories where I load the most items into a single prompt and best where the prompts are small and single-purpose. A script measures both weekly.

The third thing that survives is refusals living in the repository rather than in the agent's harness: a pre-commit check, a dry run that defaults on inside any script that spends money, a test that fails. Those survive a vendor change. A hook in one tool's configuration does not, and my agent moves to a different vendor on the 20th.

Grade artifacts that exist and tests that pass. Never grade an agent on the words it uses about itself, because vocabulary is free to change and the grading will pay for exactly that change.

Calibrate any automated control against real history before you switch it on, and print its false-positive rate next to it. A control with no number attached is a wish. If my eleven typed copies of one line had been run through the detector first, the dominant class would have died before it charged a single point.

Assume the instrument is wrong before you assume the world is. A zero and a suspiciously clean series both deserve a second measurement. Mean deductions of 0.0 in both arms of a test was the tell, and it sat in the output for days.

Put the human at the merge and at the spend. My time is worth something in both places. It is worth nothing adjudicating whether a close-out was phrased well.