Hi there,
Welcome back to Untangled, a newsletter and community for people who refuse the story that technology just happens to us. It’s stewarded by me, Charley Johnson, and enriched by members like you.
This week I’m sharing my conversation with Paula Goldman about her new book, Manage the Machine, and why rolling out AI one employee at a time misses where work gets done: between people.
As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.
On to the show!
Untangled HQ
AMA: On November 11 at 10:00 am PT, I’m hosting an AMA on organizational AI adoption. Are things going sideways inside your org? Join us!
New Tool: This week’s organizational tool is “Buying AI: The AI Procurement Card” — it walks you through what to ask and what to listen for when you’re paying for a subscription from a major provider.
Deep Dive
AI adoption happens between people. Measure it there.
I’ve argued before that “human in the loop” functions more like a permission structure than an actual safeguard. On the podcast this week, Paula Goldman—Salesforce’s first Chief Ethical and Humane Use Officer and author of the great new book Manage the Machine—explained how the phrase got so watered down. Born from a Cold War-era tradition of thinking about human accountability in complex systems, the concept has recently been flattened into something Paula dryly describes as: “AI drafts something, and I hit yes.”
She prefers “human at the helm.” You set the guardrails the system can’t cross, anomalies escalate to your desk, and the tool sharpens your judgment. Then she added an important caveat: “Or sometimes humans, plural, at the helm.”
This post is about the plural. Almost every organization I work with is rolling out AI one isolated employee at a time—encouraging individuals to experiment and counting how many people used the tool this week. Paula sees the exact same pattern: Here’s a tool. Go use it. We want to measure your individual output.
I think this is a massive category error. Because as Paula puts it, “work happens between people.” A functional team is a fundamentally different organism than a pile of individuals who just got 20% more efficient. And a dysfunctional team forms when we focus on individual output while ignoring the group patterns and dynamics individuals create.
Multiplayer Alignment?
Ethan Mollick isn’t so concerned. In his recent post, “The Dot and the Swarm,” he argued that we don’t need to worry about how AI work gets organized. Initially, Mollick assumed we’d need to design complex management structures for AI agents. Now, he thinks he was wrong: “It turns out that organizing work is just one more thing AI can learn to do.”
Mollick contends that your agent (e.g. SpaceX’s Grok Bot, Meta’s Muse , OpenAI’s dots, etc.) can now coordinate other agents to solve your problems. Not unlike the ‘swarms’ OpenAI launched to produce the proof for the Navier-Stokes equations by exchanging 2.7 million messages over 88 hours with remarkably thin oversight.
How this coordination plays out in organizations hinges on the principal-agent problem. This is the idea that people have personal goals that don’t always align with the company’s goals. “A lot of what we call management,” Mollick writes, “exists to solve problems that come from organizations being made of people.” Managers, in short, help align individual goals to the team’s goals, and from the team’s goals to the company.
Now, Mollick concedes the principal-agent problem doesn’t vanish; it just moves to the space between the swarm and us. But he’s confident that it is manageable “as long as humans are guiding them in the right direction” and so long as the agents are “properly aligned to our needs.” But who is “our”? And what does “aligned” even mean in this context?
I’ve argued previously that “AI alignment” is a myth. The systems AI touches are complex and unordered; they change as we act on them, and so do our goals. In those environments, what we actually need is an ongoing, collective sense-making process across diverse perspectives.
I still think alignment is an incoherent concept, but when I critiqued it three years ago, I was effectively writing about one principal and one agent. Now that game is becoming multiplayer in both directions -- multiple people managing multiple agents in service, ostensibly, of a common goal. Imagine a foundation team where everyone has their own agent coordinating still other agents. The program officer wants to fund a grantee she’s championed for two years, so her agent drafts a glowing recommendation. The finance lead needs to keep next year’s budget flat, so her agent flags the grant as too expensive. The comms lead promised the board a splashy announcement, so his agent rewrites the summary to exaggerate the grantee’s reach.
Each agent perfectly and faithfully serves its owner’s immediate needs. The result? An incoherent recommendation no one on the team actually wrote, born from invisible negotiations no one had, serving a collective goal the team never explicitly discussed. That is the principal-agent problem gone multiplayer that Mollick doesn’t address.
Relationslipping
This is easier said than done, since ‘the team’ is constantly evolving. A healthy team operates on what the late social psychologist Dan Wegner called a transactive memory system: a shared map of who knows what. You don’t need to hold all the expertise, because you know which teammate to ask. New research suggests individual AI adoption is undermining that map.
In a recent Harvard Business Review study, researchers surveyed over 1,000 U.S. desk workers. They found that as AI usage increased, so did the perception of teammates’ specialized knowledge. But the one metric that flatlined? How well people actually knew where that expertise lived.
When you skip a five-minute chat with a colleague to get a quick answer from an AI, you never update your mental map of their expertise. As one respondent noted, “I have absolutely no idea what [my colleagues] are thinking or doing because the generative AI does most of their work.”
The authors argue that over time, this leads to “relationslipping” (a clunky academic term for a very real problem): the slow decay of working relationships as we swap out our peers for prompts. And they found that on teams where this is the norm, coordination and trust drop over 20%.
The authors also found that people with low psychological safety on their team were more than twice as likely to turn to AI instead of a teammate. As a regulatory manager in pharma put it, “I use AI when I do not want to be thought of as an idiot or unintelligent.” On a team where asking feels risky, that’s a reasonable choice. Multiply it across a team, though, and the shared map disappears.
Designing for the Team
In Manage the Machine, Paula tells a story about social workers in an English town who trialed an AI note-taker and surprisingly rejected it. The bot worked just fine but they enjoyed their afternoon walks together, where they traded notes, vented, and worked through hard cases together. In Wegner’s terms, those ‘inefficient’ afternoons were the glue holding their team’s memory together. Leaders need to identify these so called critical inefficiencies before they automate them away.
The team is the unit that makes sense of an unpredictable world, and it does so through messy, inefficient, sometimes hard conversations. When I asked Paula about this, she pointed toward a different architecture: design the agent to shape and support the group’s dynamic. Some companies she studied programmed a team-level agent to play devil’s advocate, or to interrupt and say, “Here are the criteria we agreed on last week—are we skipping them?”
That agent works for the team. The agents aren’t coordinating among themselves to support the individual; they’re working to support the coordination and collaboration of the team. I’m into it!
But notice th…
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit untangled.substack.com
Flere episoder fra "Untangled"



Gå ikke glip af nogen episoder af “Untangled” - abonnér på podcasten med gratisapp GetPodcast.








