AI Systems, Week 2: How much does the interface shape the model?

A night-shift verifier bot that never got approved, Anthropic's filing at what may be its best moment, and protection as a subscription versus a public guarantee. Plus, from the archive: what to measure while compounding has not shown up yet.

Train the decision point: when to stop thinking, which harness to borrow, which model to call; LoCo: same-prefix preference pairs teach a reasoning model when to stop; Harness-S: sibling harnesses rescue failed RL groups, then the skill moves into the policy; Route Anything: oracle headroom says routing pays in cost before quality; System-1 gate: predict, gate, execute, update, with savings set by how rare useful actions are; Pandora: separate what a simulated user feels from how they say it; Kling: captions as a contract and a trained prompt rewrite layer; Judge check: compare AI and human rankings at the top, not on average; Claude Mods: middleware hooks and the attenti

A trusted hairdresser versus a film budget in climate communication, a hiring crisis that may be about remote work, the access fences labs build around risky models, the chip market splitting for agents, and what error bars hide. From the archive: twelve years of advice on whether to own one channel or two.

Score the net, not the gross: memory, training data, and benchmarks judged against the plain baseline; Memory tax: agent memory must beat naive RAG over raw history, and often does not.; Benchmarks as difficulty measures: measure mismatch and coverage gaps hide what a score really says.; Decision grounding: train only on actions the agent could justify from what it saw.; Turn-level credit assignment: agents decide in turns, so RL should assign credit per turn.; Interface lock-in: fine-tuning one answer format improves only that readout.; Decision models as routing economics: send yes/no agent steps to cheap typed-answer models.; In-situ interventions: a few dozen examples

Week 2 of AI Systems: Models, Harnesses and Agents. Why two tools that can make the same edit can set a model very different problems: tool contracts, trajectories, the failure points in tool use, capability versus elicitation, and why the error a tool returns is half the interface.

Why coding agents ace the benchmark and flunk the job, what happens to content when the reader is an agent, three ways to handle a tax-loss problem, dosing versus rewriting aging, how early hires get picked, and an archive thread on engaging versus excluding in conservation.

Guarantees come from structure, not from asking nicely; Waypoint-gated distillation: transfer only the RL shifts that early and late training agree on; Decision-native models: null-aware, order-proof, scalable, and cacheable by construction; Thinking inertia: no-think switches leak reasoning, so measure answer-only compliance; Demo versus delivery evals: held-out acceptance, business-uplift scoring, priced clarifying questions; User-modifiable harnesses: users edit the agent loop, subagents, and routing; Cache-aware economics: stable prefixes first, judge models by cost per completed task

Week 1 of AI Systems: Models, Harnesses and Agents. Where an agent's behaviour comes from: weights, context, inference compute, and the runtime that executes its actions; how to attribute a behaviour with controlled interventions instead of guessing.

A chip maker pays eight billion dollars for a model that invents camera angles, two AI labs disagree about who holds the agent's leash, colleges face arithmetic that has nothing to do with AI, and an archive thread on thinking with the body.

The signal is narrower than it looks: recency posing as memory, failures that teach, less supervision that wins; Memory-need stratification: pooled memory scores mostly measure recency; Memory framing: calling context "memories" changes how often it gets used; Hindsight-to-foresight distillation: failed runs teach when group rewards are flat; Reliability over coverage: distill only where tokenizers align cleanly; Experience retention: raw logs versus summarized playbooks for learning agents; Cheap discriminative judges for closed-answer evals at volume; Brain and hands split: editable harnesses, cloud reasoning, device actions; Action gating: approval policy plus readable