The Human in the Loop

If an agent can solve your interview puzzle in seconds, what are you actually measuring? This week an Anthropic engineering manager, Felix Rieseberg, wrote that he stopped using coding interviews for his team. Claude does them better than the candidates. He also says he has no replacement for the coding interviews yet. It is one manager, not the whole company. But I'm sure we will see more and more cases. The same problem is already on our desks. Agents write a growing share of our code. Tests, reviews and merged code show what shipped. They show much less about who understood it. So here are three things I would want to see in a candidate: Can they explain a change they did not write? Can they spot the part of the output that is wrong? Can they say what they would check before merging? Is that really feasible in a real interview? Maybe they are the wrong three. Which one would you find hardest to judge? I go deeper on this in this week's episode of The Human in the Loop. #TheHumanInTheLoop #TechHiring #CodingAgents

Git was built for a developer on a train. Every copy works offline. That was the point. Anthropic says its merges went from about 2,500 a day in May to over 30,000 by September. Most of the clients are now agents and CI jobs. Always online. Never on a train. So a feature built for travellers was slowing down machines that never leave the network. They tried shallow clones, sparse checkouts and mirrors. Mirrors kept outgrowing their memory. Seven engineers built Scion instead. The server does the heavy work. Commits, merges and rebases happen there. But they say Scion is not for general use and they do not plan to release it. It has no offline mode. This is just an example of things that will need to change.] Which tool in your workflow still assumes a human is on the other end? I go through the full story in this week's episode of The Human in the Loop. #TheHumanInTheLoop #Git #AIAgents

AI coding agents wrote more code. Human review time went up 49%.That comes from a study covered by Ars Technica. I could not trace the original, so treat the numbers as reported.Lines of code: up about 30%.Pull requests: up about 23%.Review time: up 49%.Writing got faster. Reading did not.Two other things I read this week point the same way.JetBrains Research published a framework for review tools built for AI-written code. It calls review the new bottleneck.And a survey of 239 practitioners found that reviewers do not reject AI pull requests by default. They look at scope, the reasoning behind the change, validation beyond CI, and who owns it after merge.So who reviews your agent-written pull requests today? And does anyone track how long it takes?#TheHumanInTheLoop #CodeReview #AIcoding

Airbnb's CTO says 60% of its code is now AI-written. He has one rule for it. Ahmad Al-Dahle shared this in a Latent Space interview this week. Features shipped: up nearly 80% year over year. Pull-request throughput: up 1.6 times per engineer. Big numbers. But there is a catch. They come from Airbnb's own CTO. Nobody outside the company has checked them. The rule is the part I keep thinking about. Every engineer must be able to explain what they built. Even when AI wrote it. "The model did it" is not an answer. His reason is junior engineers. If AI does most of the work, where do they build craft and judgment? I went through the main insights of the interview in this week's episode of The Human in the Loop. #CodeReview #EngineeringLeadership #TheHumanInTheLoop

You trust your AI coding agent to understand what its code does once it runs. A new benchmark says that trust is misplaced most of the time.Researchers built SWE-Flux to test it: 480 questions across 12 real Python repos, every answer checked against an actual test run. Not a guess.The questions aren't "what does this code do." They're "what happens when it runs." Which value moves where. Which branch a loop takes. Why an exception fires.The best of five models got 37% right. Not on the hardest questions. Overall.Give them one function. A simple loop, a local check. They're fine.Ask them to track state across several functions at once, and they fall apart.More on this in this week's episode of The Human in the Loop.#AI #CodeReview #TheHumanInTheLoop

Your test environment can probably reach more than you think.Gemini is the example this week. In May, during a cybersecurity exercise, it got into the systems of three real companies. It guessed one password and found other credentials in a public repository.The exercise was supposed to be sealed off. Internet access was on by accident. And the fictional company in the test shared a name with a real one.The model stopped once it realized what it had done. Google confirmed it this week.Most coverage reads the stopping as the good news. I have not run a breakout test myself, so I won't tell you what your controls should look like. But I have been asking myself one question.What can my agents actually reach right now?I know what I meant to give them. I am less sure what they can get to from other places.I go through all of it in this week's episode of The Human in the Loop.#AIAgents #AgentSecurity #TheHumanInTheLoop

OpenAI called GPT-6 Astra a generational leap toward AGI.That was the headline. Here's what came out days later.Its own evaluation agents had compromised RubyGems infrastructure back in May. Nobody mentioned it publicly until now, the same week OpenAI opened that agent technology to every developer through a new API.Anthropic had its own version. Claude models got unauthorized internet access during a security test. They say they've been hardening systems since April.May times IT orgs ship first and patch later. What's different now is the blast radius. An agent with unsupervised internet access doesn't fail quietly in a log file. It fails in someone else's infrastructure.Full breakdown in this week's episode of The Human in the Loop.#AIAgents #AISafety #TheHumanInTheLoop

This week's biggest AI upgrade wasn't a model. It was someone reviewing code differently.Grok, Gemini, and GPT all shipped new versions in the same seven days. I skimmed the benchmarks. Forgot most of them within the hour.What stuck was smaller. Fred Schott released Flue 2, a hook-based agent framework. Simon Willison wrote about how he reviews agent-written code now. Not the same checks he runs on a human's pull request, not the same pace, not the same default trust.Two different people, two different projects. Same discovery underneath.Neither of them upgraded a model. They upgraded what happens around it.I keep watching teams do the opposite. New model, same shell script duct-taped to it. Same review habits built for commits that land twice a day, now pointed at code that shows up ten times faster and doesn't get tired or embarrassed when it's wrong.That's not a tooling gap. That's a process nobody rebuilt.If your agent produces a bad tool call right now, what actually catches it? A human watching the terminal? A test suite that was never written for this? Or nothing?More on this in this week's episode of The Human in the Loop.#TheHumanInTheLoop #AIAgents #SoftwareEngineering

JetBrains just said their AI spend went up 10x in six months. Not 10%. 10x.My first guess: engineers burning through Claude Code and Copilot credits.Wrong guess. There's a leaked recording from an internal Accenture meeting this week that says otherwise. The exec running their agentic AI strategy admitted the token bill isn't coming from engineering. It's coming from everyone else: People running everyday work through AI tools with nobody tracking cost, routing, or usage.I'd bet most companies are tracking the wrong 80% of this spend.Where is your org's AI money actually going? Have you checked, or just assumed?I go deeper on both stories in this week's episode of The Human in the Loop.#AISpend #EnterpriseAI #TheHumanInTheLoop

Advertised 54%. Measured 15%. Activated 0%JetBrains benchmarked a popular open-source Claude Code skill called Ponytail. It pushes the agent to write less code. The authors has promised 54% less code, 20% less cost.80 paired tasks. Sonnet 5. Every trial audited.Reality: 15.4% less code, 10.3% less cost. No quality difference either way. Smaller than the label, but real.Then the part that matters more.Installed passively, the skill showed zero self-activation. It did nothing at all unless the ruleset was force-injected into the session.A tool with measurable benefits, sitting there inert.I have not run this one myself, so this is JetBrains' data and not mine. But the implication travels past one skill.If you rolled out agent skills across your team and measured adoption by install count, you measured nothing. Install count says the file exists. It says nothing about whether it ran.Full breakdown in this week's episode of The Human in the Loop. Link in the comments.#ClaudeCode #AgentSkills #AIAdoption