Where a Person Decides: Human in the Loop, by Rule
In 60 days, our AI judge passed 14 of the 19 jobs a person rejected. Why a person still decides, the question the judge never asks, the rules that call a person in, and what a decision becomes.
Writing
Essays and open-source releases from the team. No hot takes, no vendor pitch — just data and patterns from systems we run ourselves.
In 60 days, our AI judge passed 14 of the 19 jobs a person rejected. Why a person still decides, the question the judge never asks, the rules that call a person in, and what a decision becomes.
Read →How a system learns from its own work without anyone typing the answer, in reading order. Ten pieces; the Knowledge Layer is what they add up to.
How the system around the model is built: gates, scaffolds, context delivery, and why structure beats instruction.
In 60 days, our AI judge passed 14 of the 19 jobs a person rejected. Why a person still decides, the question the judge never asks, the rules that call a person in, and what a decision becomes.
An AI agent retrying a failed step can loop forever, and a fixed cap only hides it. How BPOS decides after every pass whether another is worth paying for, and why a bounded loop is not a converging one.
Every BPOS node chooses what to read before it acts, in a turn whose whole output is that choice. Three side-task attempts that scored zero, what the separate turn changed, what it costs, and what it should read.
Reviewers rarely reject agent code for being wrong; they reject it for not reaching far enough. How BPOS finds the blast radius of a change by descent, and why create and edit are one operation at different distances.
Nodes that declare what they listen to, what they publish and what they need from each other, fire in waves, and ask peers for pointers instead of paraphrases. And the neuron analogy the design follows.
A job that passed can still have passed by luck. How BPOS's epistemic round asks why every outcome happened, successes included, keeps only answers that verify something, and proposes changes to the pages that should learn.
A quality gate for AI agents, rule by rule: criteria judged three times by an LLM judge, a failed declared check scoring zero, the platform sending back its own failures, and a person deciding the rest. With the flaw we have not fixed.
What procedural memory is, where AI agents usually keep it, and how BPOS keeps it in skill pages and the links between them: learned by practice from every outcome, with the traffic showing what agents need, and every step readable.
What a context graph is, why decision traces matter, and how Lite keeps facts, rules and decisions apart: conflicts put to a person, decisions that supersede with their history, everything written as the AI works. In beta, by invitation.
SKILL.md is a good, portable recipe for agent skills. What a BPOS skillset adds: tools, model and answer form declared per stage, links that learn from every outcome, and a server log of every search, hop and miss.
In BPOS a node becomes an agent for one stage of one job, from the skillset the job carries, and ends with it. Why we keep what agents did instead of the agents, and how that record teaches the next ones.
What a BPOS job contract holds, how the planner writes it, the COCA frame that carries it into every prompt, how it is judged three times, and where our contracts are still weak.
Some stages of a goal cannot be skipped, whoever does the work. How BPOS writes them down as a program, how Lite was built through one, what a necessity is, and where we are behind the research.
For a year we wrote about what we learned, not about the machine that taught us. This is the machine: how a goal becomes shipped code, what decides what ships, and what it does not do.
Google's SDLC paper says the work is moving to specification and verification. A second Google paper says a hand-written specification can make an agent worse. Both are right, and the gap between them is where we work.
A wiki is knowledge you go to. Working knowledge is knowledge you look through. The difference decides whether an AI agent uses what your company knows, or ignores it.
Three kinds of claim live in every company. A knowledge base that stores them as one kind will learn nonsense, and so will the AI that reads it.
A soap film, a slime mould and a pigeon each add one thing to a rule that learns. The law BPOS uses, why it resists superstition, what our own graph has actually earned, and the evidence against us.
We decide how BPOS learns. The work decides what it learns. The three ways a learned weight can move, why one is locked, and the eight days that taught us.
Given no guidance, an AI agent scored 87.50 on a benchmark task. Given an expert’s map of the task, it scored 58.93. What repaired it, and where our own system still falls short.
A passing pipeline proves the code compiled and the tests you wrote passed. It does not prove the work does what was asked, or say why it worked. Across 12,103 recorded gate decisions and 331 sourced rejection notes, that difference is where AI-written software quietly fails.
Reasoning traces from a third-party API are a safety surface you rent, not own. Building enforcement on them is faith, not verification. Three beats from our own stack: a trace reader, dependency ordering, and a portable test that survives a provider change.
We put a mechanical grounding gate in front of our plan generator and watched violations fall from nine to zero across seven generations — without loosening the gate once. The last two defects lived in the generator's own scaffold and in the judge itself.
We bet the system around the model matters more than the model itself. Over the last few months, five independent practitioners and a frontier lab arrived at the same conclusion — none of them ours.
Lazy skill loading for agent systems. Claude Skills are useful when loaded on demand, expensive when loaded up front. context-steward is the npm package that solves the second problem without breaking the first.
Multi-agent delivery doesn't break on the model. It breaks on file conflicts, bad sequencing, and per-job metrics that miss the deployment. The unglamorous machinery that decides whether a sprint ships.
Measured findings, including the ones that said no. Nulls published next to wins — the method is the product.
An AI assistant told us our system could not do eight things. Seven of those limits were never there. How to tell a perceived limitation from a real one before you build a workaround.
LLM-as-a-judge has a failure mode nobody warns you about: saturation. Our conformance judge returned exactly 1.00 on roughly 92% of non-failed runs, looked statistically precise, and scored a fabricated output and a faithful one identically. The measurement, and the vector that replaced the single score.
One incident, fully instrumented: a reviewer wrote a plausible causal theory into rejection feedback, the model obeyed the theory, and the retry regressed. Why the feedback channel carries sourced facts only. An n of one, stated as one.
We built skill checklists for our agents, believed in them, and measured them anyway. Across 88 arm-randomised scored executions, quality moved +0.002 — inside the noise. The null, the method that caught it, and the boundary on what it does not say.
A running record of independent practitioners, an academic study, and frontier labs arriving at the conclusions we built on — in their own words, with dates, sources and caveats. Updated as the record grows.
We audited 36 mechanical quality checks and found about 20 had caught nothing in a month of production. The scaffolding you build for a weak model becomes a cage for a strong one.
A GDPR-certified consent tool got a client fined anyway — because blocking a tracker doesn't make you invisible, it makes you a specific, reconstructable kind of missing data. On MNAR statistics, Bayesian imputation, and what enforcement actually catches.
Per-job quality scores in multi-agent systems almost always land around 85%. That number is telling you something different than you think it is. 4,600+ scored executions from our own production, and the pattern we found.
We ran a controlled A/B across 88 scored agent executions, with and without skill checklists in the prompt. Quality moved +0.002 — inside the noise. Why front-loaded instructions don't land, and what does.
Listening to production: telemetry, failure surfacing, and the feedback loops that turn incidents into structure.
Our supervised agent platform ran a night of maintenance on itself. Five defects came out, zero of them model failures. The strongest warning in the system was written by a worker, for free, into a table nothing reads.
Server-side click attribution: record the click at the link, before the browser can drop it. How HikrLink does it, the one thing it refuses to do, and the three wiring bugs we found joining it to our own tracker — none on the link side.
SEO as an instrumentation problem: four sensors, deconvolution that checks its own arithmetic, a hysteresis machine cleared through a four-run backtest, and a publisher that mechanically says no. Every decision leaves a receipt.
The same failure surfaced twice in one day: 47 minutes in the morning, 6 seconds in the evening — a ~470× difference. Five changes, one principle: listen to the system's own signals.