Episode 8 - Agent Orchestration (Track A - System)

Ninety-two percent accuracy. A green arrow on the slide. And users quietly abandoning the product underneath it.An insurer's claims-intake agent reported beautiful model metrics for two months while adjusters drifted back to reading files themselves — the errors hid in the complex claims, nobody could tell which fields were wrong, and "good" was never defined in the adjuster's language. The root cause wasn't measurement. It was definition, inherited from the vendor dashboard that shipped in the box.This episode lays out the fix: four numbers to write before the first prompt. Task success defined in the user's units and verified in the system of record. The escalation band, which fails in both directions. Cost per successful outcome, not per call. And the harm line, the wrong action that must essentially never happen. Plus who signs the bar (the business owner, before the build), why the four numbers work as tripwires that police each other, the escalation band as the leading indicator, and the floor you write down while you're still clear-headed.With a worked example (the four numbers for the bank payments agent from The Not Yet List) and the win: a retailer's returns agent whose operations director wrote the definition of good into the kickoff document, above the architecture diagram.This is Track B, Episode 3 — the Product track: what to build, and why. New here? Start with Episode 1: Beyond the Demo.Monday move: finish the sentence "this agent is good when…" in the user's units, write the four numbers and the floor, and get the business owner to sign it. If nobody will sign, you don't have a metrics problem. You have an ownership problem.

Fourteen service lines. One prior authorization agent. A go-live that lasted six weeks.The episode opens in a hospital — a health system that gave one agent all of prior auth on day one, every service line, every exception — but the story isn't really about healthcare. It's about the shape of every failed agent deployment: an agent asked to be trusted with everything, everywhere, on day one, that guessed when it should have asked for help.Then the mirror image, in financial services: a regional bank's payments agent that shipped doing exactly one job — investigate failed payments, show the evidence, move no money — survived two weeks of analysts trying to catch it being wrong, caught a small business's payroll file before it bounced, and earned each expansion after that.The argument: version one of an agent is not an MVP. An MVP tests demand; version-one scope is a trust decision with a blast radius. Why smart teams overscope anyway (demo pressure, stakeholder arithmetic, the vendor platform story), what actually counts as "one job," and the discipline that fixes it: the not-yet list — everything the agent deliberately won't do on day one, written down, read out loud, with an evidence bar for how items earn their way off. Scope expands by grant, never by drift.This is Track B, Episode 2 — the Product track: what to build, and why. New here? Start with Episode 1: Beyond the Demo.Monday move: write your agent's not-yet list — five things it will not do on day one — and read it out loud to your sponsor before the next build review. If the list is empty, you don't have a scope. You have a hope.

Episode 8: Orchestration — Layer two of the nine-layer architecture: orchestration. Who decides what happens next.We open with a five-agent insurance claims crew that did everything right and still went sideways, then work through what the major labs actually agree on: start with one agent, bound the loop, let code own the known path, and spend the model's judgment only where the decision is genuinely uncertain.Inside: the ReAct loop and its guardrails, the supervisor pattern, agents-as-tools versus handoffs, the four orchestration shapes, what should travel on a handoff, Anthropic's hard numbers on multi-agent cost versus value, and why delegation quality is the number one failure mode. Plus a second war story: the Tuesday a support triage chain fell over.Monday move: mark every decision point in your most complex agent flow as model-decides or code-decides, then encode one deterministic decision as code.

A refund agent with a human approval on every action sounds safe.Until reviewers are clicking approve half-asleep, and a $4,800 decimal slip sails through in under two seconds.In this episode: the handoff layer. Where the agent meets the user, and who's actually in charge at the moment something irreversible happens.When should the agent ask vs. act? (One agent guessed wrong and ordered noodles instead of milk tea.)How to gate without alert fatigue: stop where money moves or something can't be undone. Never on read-only work.Why transparency is what protects user autonomy.Plus a beat for PMs: the approval dialog, the escalation message, the activity log. That's product surface you can own and prototype yourself, starting now.Track A, Episode 2 — the System track: how you build it.New here? Start with Episode 1: Beyond the Demo, or catch the Two Tracks interlude for the full map.Monday move: take your highest-stakes agent action and list every approval it triggers. Cross out each gate that doesn't map to something irreversible or expensive. For every gate that survives, rewrite the user-facing sentence until a reviewer can make a real decision from it.

🎁 Free download: The Agent Brief template from this episode — get it here: The BriefSix weeks of building. Demo day applause. Then the pilot died quietly — because nobody had written down what the agent was supposed to do.Alex and Jordan open Track B — the Product track — with a support-triage agent that auto-resolved a billing ticket it got wrong. The post-mortem asked the obvious question; nobody had an answer on paper.The discipline: two pages before anything gets built. Page one, the announcement, written as if the agent already shipped. Page two, the FAQ of dreaded questions — what won't it do, what happens when it's wrong, who approves what, when do we kill it. A brief that gets rejected is a success: you killed the wrong thing on paper in an afternoon instead of in production six weeks later.Every dreaded question gets its own episode this track. By the end, your brief is done — and the agent is actually buildable.This is Track B, Episode 1 — the Product track: what to build, and why. New here? Start with Episode 1: Beyond the Demo.Monday move: write page one for your most important pilot. Read it aloud — if two reasonable people could build two different agents from it, write it again.

Every agentic system is really nine layers — whether you designed them or not.In this episode, Alex and Jordan walk all nine through a real war story: an invoice agent that worked in the pilot, then paid twelve thousand dollars to the wrong vendor in week three of production. The agent did everything it was told. The problem was everything nobody told it.The nine layers: the handoff (experience), orchestration, decision intelligence, integration, context and memory, the data foundation, the model layer, security and trust, and observability — with governance cutting across all of them.The thesis: the layers aren't the hard part. The boundaries between them are. Two skipped decisions — nobody defined what the agent may conclude, nobody owned the data it read — cost twelve thousand dollars.Plus, a beat for the product managers: you don't need to draw all nine layers yourself, but you need to know which layer your requirement lives in.This is Track A, Episode 1 — the System track: how you build it. New here? Start with Episode 1: Beyond the Demo, or catch the Two Tracks interlude for the full map of where the show goes from here.Monday move: take your most important pilot and draw the nine layers on a single page. For each layer, write the one decision you actually made. Where it's blank, that's your architecture debt.

Episode 4 asked what to build. Episode 5 asks how to build it. This is the bridge between them.In this short interlude, Alex and Jordan announce the show's new structure: two tracks running in parallel.Track A — The System. How you build it: architecture, data, evals, governance. Starts with Episode 5: Architect the System.Track B — The Product. What you build and why: the brief, version-one scope, metrics before building, shipping to people who don't trust it yet, and knowing when to kill it.Why two tracks? Because product managers are becoming builders and builders are becoming product thinkers — and the old handoff between them doesn't survive contact with agents. Every system episode now carries a beat for the PM; every product episode carries a beat for the builder.New here? Start with Episode 1: Beyond the Demo.

Most agentic pilots don't die from bad engineering. They die because nobody wrote down what "working" means.In this episode, Alex and Jordan take the product line: the agent is a product, not a project. It has users, an owner, and a definition of done.In this episode:- The product brief: one paragraph answering four questions before anyone maps a process- Users are plural: the end user, the supervisor, and the auditor, and why all three belong in the brief- Define what good looks like first: accuracy, speed, and escalation rate, picked while you're still honest- Version-one discipline: the most powerful list in the brief is what the agent will not do yet- Name one owner: every agentic product that survives has one, and every one that dies was owned by a slide- Then dive into the process anyway: end to end across the silos, because the agent inherits whatever you don't mapThe Monday move: take one use case and write the one-paragraph product brief. Who uses it, what job it does for them, how you'll know it worked, and what's not in version one. Then trace the process it sits on, end to end. If you can't do both on one page, you've found your first problem.

Before you listen to a single episode, listen to this. Two minutes on what The Agentic Enterprise is, who it's for, and the playbook we're building together.In this episode:Why this series exists: cutting through the AI agent hype for practitionersThe gap it lives in: what works in a demo versus what survives in productionWhy practitioners: the people who feel the consequences when agents failWhere it comes from: more than twenty years in the field, hundreds of systems built across consulting, technology, and bankingThe season goal: a complete practitioner's playbook, one episode and one Monday move at a timeYour first move: start with Episode 1, "Beyond the Demo" — it begins with the most abused word in the industry right now: AI Agent.

Where do the agents go in — and where do the humans stay in charge? Alex and Jordan lay out the map-before-you-build discipline for placing every step before a single bot is built.In this episode:Start with evidence: process mining and system logs — if the map doesn't surprise you, you mapped the brochureWalk the real process: documented versus real, with the SMEs who live in itDecision logic: deterministic gets encoded as workflow; interpretive gets AI assisting the human firstSupervision and risk: the agent proposes, a person disposes — with risk and legal as co-designersMap as contract: every agent insertion point gets a testThe Monday move: pick one process, map the real current state with two people who do it daily, then mark three things on the future state: where agents go in, where humans stay in charge, and what risk and legal need to see. One page.