Back to the network

Tooling

ai-automations

What runs while no one is typing: an overnight runner whose agent jobs work in a jail behind a guard that fails closed, 33 jobs, weekly reviews and local-first tools.

A review that has to pay for itself

The weekly efficiency review asks one question: how could Claude, the local models and the free cloud models do every task I do better and more cheaply? There is no bill to cut. The budget is my plan's usage limits, the free models' quotas and my own hardware, so the review measures those.

It runs in four stages: collect, watch, review and verify. Three of them are plain code at no cost. Each machine writes a weekly scorecard of numbers only, with no project, branch, session or file name and no prompt text. The watch reads a fixed list of public sources, such as Claude Code's changelog, Anthropic's pricing page, the free model lists and new local model files, along with what 1-llm serves, and compares each with its last snapshot. A source that fails is not a source that emptied: it keeps its last snapshot and compares nothing that night. A first snapshot is a seed, not a change. The watch never acts on anything it finds.

Only the review stage calls a model, and it reads numbers and differences, never prompt text. It runs diff-first and model-last: a quiet week, with no material change and no metric past its threshold, skips the model at no cost. When it runs, it is one Sonnet call with no tools, and every call of the review, its retry included, shares a cap of $1 a week. That cap lives in the script, because a job file's dollar limit does nothing for a script job.

The check before I see a proposal

The review answers with a list of proposals. Each one says what to change and where, what kind of change it is, the effect it expects with the arithmetic behind it, the quality metric it could hurt, the scorecard values it cites, and how to verify it: a metric, a threshold and a number of weeks.

Code checks that list before I see it. Every value a proposal cites has to be in the week's scorecard and match within 1%. Every metric it will be judged by has to be normalised, a share or a value per commit or per active day, because raw weight is not efficiency. Every file it names has to exist, every model a routing change names has to be one that is served, and nothing in it may hold a work name or a secret. A list that fails gets one retry with the errors attached. A second failure sends the scorecard line to the morning brief with no proposals, and the reason.

A proposal, from my answer to its verdict

The life of a proposal from the weekly reviewA proposal starts open, once its evidence has been checked, and waits for my yes or my no. A yes accepts it, and a lane or I make the change. A no rejects it, and four weeks with no answer expire it; either way the same change is not proposed again for eight weeks. Once the change is in, the proposal is adopted and the mean of its metric over the last two weeks is stored as its baseline. After the weeks it names, the metric either passes its threshold, and the proposal is held, or misses it, and it is not held. A held proposal is checked again every four weeks, drawn as a dashed line, and regresses when it no longer passes.A proposal, from the review to its verdictOpenthe review's proposal, its evidence checked;it waits for my yes or my noAcceptedmy yes, by its id;a lane or I makethe changeRejected, expireda no, or 4 weekswith no answer; notproposed again for8 weeksAdoptedthe change is in;its metric's mean ofthe last 2 weeks isstored as baselineHeldafter its weeks, themetric passes itsthresholdNot heldafter its weeks, themetric misses itRegresseda held change thatfails its re-check
A proposal's life in the review's ledger. I accept or reject it by its id. Once the change is in, the metric it named is measured against the baseline stored that day, and a held change is checked again every 4 weeks.

A rejected or expired proposal is not made again for the same kind of change in the same place for 8 weeks, so the review cannot nag. A week in which a metric measured nothing, such as a week with no active day, is left out of its mean, and a proposal with no measured week waits.

Its own test

For its first 8 weeks the review only proposes, and I decide each proposal by its id. At week 8 the report counts, for each kind of change, how many held and how many did not. A kind with at least 3 held and none failed qualifies for drafted branches, a mechanism a later lane builds only if some kind qualifies.

From week 8, every 8 weeks, the review tests itself. With no adopted proposal, the report says "no adopted proposal, no verdict", and its model stage stays on, since it cannot be blamed for what was never tried. With at least one adopted, three things must hold: the held proposals save at least 10 times what the review itself cost, at least one proposal held, and the quality guardrails, red nights and rework, are no worse than in the 8 weeks before. If the test fails, the model stage switches off until I remove a file, and the brief says so.

The review was built on 2026-10-20, so it has not reached week 8, and this page reports no verdict.

The automation finder

Once a week, the finder looks for what I do by hand again and again, or ask Claude to do again and again, that could run without a session. It reads the last 28 days of three sources: Claude Code's transcripts, my shell history, and the transcripts of my screen recordings. Every line goes through the secret scrubber's redaction in memory before anything parses it, and the report, its data and every model call are built from redacted text only.

A candidate is a sequence of commands repeated at least three times over two days or more, a prompt I typed three times or more, or a manual task I described twice or more in my recordings. Rules, not a model, pick the cheapest tier that fits: a plain script, a script with a local summary, a local model, a free cloud model, or Claude. Only for a task described in a recording does the model guess the tier, and the report marks it as a guess. Each candidate carries what it costs now and what it would cost automated.

The finder may cost $10 a week at most, and less is better, with $3 for a single run. An approval that fits the budget queues the build as a once-job for the runner: one build at a time, with no runner review, because the lane that merges the branch reviews it anyway, and no follow-ups, so its cost stays inside its cap. A candidate over a cap is still listed, with its cost, and approving it is refused: raising a cap is my decision. A rejected candidate never comes back, and neither does a new one whose set of words overlaps a rejected one's by 0.75 or more.

I decide in a session with /automations, a small command that runs on Haiku. The list is printed by the finder's own command line before the model sees the prompt, and one shell call carries my decision. A candidate whose evidence comes from work never reaches a build agent: a session of mine builds it, and the finder records it as built. The first three were settled that way on 2026-10-24: two were built, one of them suite-run, and one was rejected, since there was nothing to build.

Transcript learnings

The efficiency review owns cost. The transcript learnings own the rest of what my sessions show: quality, how well the agents understand me, and anything else worth changing. Every night, each settled session with two or more prompts gets one digest through the local-first ladder. A session from work is digested on the local model only, and keeps no text. Once a week, one Sonnet call draws the digests together and drafts at most two changes, as once-jobs whose branches a lane merges. A change to the agents' memory is proposed, never written. The efficiency review gets a file of counts from it, for its guardrail on how often I correct an agent.

It landed on 2026-10-25. Its first live week, with a report and a proposal a lane landed, is still open.

The prompt miner

Every week, the prompt miner groups the prompts I typed three times or more in the last 28 days, first by the same text and then by shared words, and drops any prompt that holds a secret. For each group, the local-first ladder proposes a card, a command or a rule. The report quotes my prompts, so it stays on the machine.

Stack

  • Python
  • bubblewrap
  • systemd
  • Claude Code

Connected to