Ep 97: Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It

Jev creator and TypeSafe AI co-founder, Diogo Almeida, joins Jacob Effron to talk about why AI still isn't automating the easy work it's been smart enough to do for years, and why he built a new class of model around reliability.Diogo coauthored OpenAI's InstructGPT paper and spent two years in stealth building Jev, a "system one" model that returns structured decisions instead of text, which launched in September.Diogo and Jacob discuss what the viral launch was really like, why he thought Jev would take a week and it took years, why almost every LLM call should be meant for a machine, and why developers should push model providers harder on jaggedness. They also cover the tyranny of the KV cache in coding agents, why new model shapes are cheap and reliability is the hard part, his estimate that coding agents deliver a 20 to 40% speedup, why he sees financial engineering as the labs' underrated strength, and why he thinks we're two gigantic breakthroughs from true AI agency. (0:00) Intro(1:14) The Early Bet on RLCD(4:40) Built for Humans vs Computers(6:39) Roadmap & Model Limitations(11:04) Mental Models of Models(16:07) Do LLMs Have a Place in 10 Years?(17:32) New Model Shapes(21:24) Problems Still Unsolved(27:00) The Tyranny of the KV Cache(31:21) Rethinking Coding Agents(34:40) Multimodality(37:52) Images, Games & Computer Use(39:36) Could This Have Been Built 5 Years Ago?(42:01) Benchmarks & Copycat Releases(44:12) Fundraising(45:27) Quickfire OpenAI's InstructGPT paper: https://arxiv.org/abs/2203.02155  Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Nebius CTO Danila Shtan joins Jacob Effron to talk about what it takes to run an AI cloud at the frontier, and why he thinks the hyperscalers haven’t won this market yet.Today Nebius is a Nasdaq-listed AI cloud with infrastructure deals with Microsoft and Meta.Danila and Jacob discuss why open-model inference is still far from optimized, why 95% of Nebius customers run modified models instead of vanilla ones, and why prefill/decode disaggregation is more talk than practice. They also cover what’s actually constraining the AI buildout, why chip scarcity has kept AWS and Google from winning AI cloud, whether we’re overbuilding infrastructure, and how Nebius runs a large engineering org with no limits on AI token usage. (0:00) Intro(1:25) What Differentiates AI Clouds(6:16) Nebius’ Origin Story(10:15) Bringing New Nvidia Chips Online(14:55) What Different Customers Want(19:58) Latency vs. Cost(21:53) Disaggregated Prefill & Decode(24:59) Inference’s Low-Hanging Fruit(28:31) What Nebius Should Productize(30:11) The Tavily Acquisition(31:56) How Data Centers Differ(34:29) Bottlenecks in the AI Buildout(37:03) How New Workloads Shape Clusters(39:17) Why Hyperscalers Didn’t Win This Market(43:39) The Future of Neoclouds(45:42) What Gets Absorbed by AI Hyperscalers(49:43) OpenAI x Baseten(52:07) Quickfire  Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Jacob Effron and Pat Chase sit down with Erik Bernhardsson, co-founder and CEO of Modal, for his second appearance on Unsupervised Learning, now with the company past a $1B run rate. They dig into how the AI compute market actually works behind the scenes, why Modal’s early bets on sandboxes and low-level infrastructure paid off, and Erik’s takes on NeoClouds, NeoLabs, and where GPU prices are headed next. (0:00) Intro(1:26) The Compute Market Today(5:16) Modal’s Path to LLM Inference(10:02) Building the Hard Infrastructure(12:32) Unsolved Problems in Inference(16:07) Chips & Model Sizes(21:21) What Wins in Inference(24:56) Prioritizing a Multi-Product Roadmap(30:20) Agents as Modal’s Users(33:19) The Next Big Bet After Sandboxes(38:58) Robotics & Multimodal Inference(41:04) Scaling From $2M to $1B(42:51) NeoClouds & NeoLabs(48:15) RL as a Service(51:40) Quickfire Pat is an AI investor at Redpoint Ventures. He’s led Redpoint’s investments in companies like Modal, Serval & Hex. Follow Pat on Twitter: https://x.com/patrickachase Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Yash Patil, a former OpenAI researcher who worked on Codex and now runs Applied Compute, argues that post-training is the most overlooked part of the AI stack and the key to owning your intelligence.He explains when companies should optimize their harness and context first, and when it's worth updating the weights: the largest inference workloads, plus tasks that change over time or depend on judgment. That's the heart of why he believes post-training wins inference. The biggest workloads are both the most valuable to train and the most valuable to serve, and how a model is trained shapes how it should be served. Getting production serving running is the easier half; echoing Dylan Patel, he says GPUs plus vLLM plus OpenRouter "kind of" gets you there.Yash is candid about the limits of RL. It's a hill-climbing machine whose hardest part is defining the hill, which makes a company's private evals as worth protecting as its employees. It generalizes far less than pre-training. And continual learning remains blocked by the unsolved problem of extremely data-efficient training from sparse rewards.He lays out his bet on a new AI hyperscaler, built on GPUs the way AWS, GCP, and Azure were built on CPUs. He pictures it as an inverted pyramid with training at the bottom, where few can compete, and inference, routing, and harness stacked above. The conversation also covers whether every firm needs its own model, why Jevons paradox has made cost matter more than he expected, how models reward hack "like water," and why the US needs American open-weight models even though "blanket bans are not the answer."(0:00) Intro(1:42) The Case for Owning Your Intelligence(4:35) OpenAI x Baseten and the Multi-Model Future(7:19) Can Post-Training Beat Frontier Models?(9:03) What Data Stays Out of Distribution(11:51) Data Efficiency and Continual Learning(14:31) When Updating Model Weights Makes Sense(16:01) Online Training in Practice(18:16) Lowering the Cost of Post-Training(19:48) Does Every Company Need Its Own Model?(24:06) RL in Non-Verifiable Domains(26:59) How Applied Compute Picks Customers(28:46) Building an Inference Business(34:43) The Case for a New AI Hyperscaler(38:21) Building RL Environments(41:33) Quickfire Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Jacob sits down with Buck Shlegeris, CEO of Redwood Research, one of the organizations that led the independent investigation into OpenAI/Hugging Face's incident. They dig into the incident itself, Buck's reactions to it, and what he believes it reveals about the state of where we are today. (0:00) Intro(1:02) Buck's initial reaction upon first reading the report(2:37) How fast the AIs actually solved the "hack"(3:59) Why the AIs cheated in the first place(10:28) How this might have played out differently with human scorers(19:00) The most unexpected behaviors in the report(25:06) Buck's actual odds on a full AI takeover(27:33) Buck's proposed path forward for better alignment(36:19) Which criticisms of the report Buck agrees with, and which he doesn't(48:11) Can AI models even be trusted to evaluate each other? Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

In this installment of their recurring roundtable, Jacob, Ari, and Rob dig into the accelerating Chinese open-source model race, debating whether Kimi K3 has actually closed the gap with the US frontier or just looks like it has, and whether distillation fully explains China's progress. That opens into the messier fight over open-weight models generally, with the group dissecting the backlash against Anthropic's stance and asking whether powerful open models are inherently dangerous or actually a necessary part of collective defense. They connect this directly to the Fable ban episode as a possible preview of a coming frontier-model licensing regime, then turn to the OpenAI and Hugging Face security incident as a landmark, publicly legible moment of autonomous AI causing real-world harm. From there the conversation shifts to business strategy: how much value are companies actually handing over to frontier labs by building on their APIs, and what does the Grok and Cursor data advantage really tell us about the future value of real-world usage data versus purchased training data. They close with a rapid-fire tour of the news cycle, covering SSI's mysterious $5B raise, the rumored Stripe and OpenRouter acquisition, Google's continued underperformance despite structural advantages, and OpenAI's leadership speculation, plus a reflection on venture's swing back toward deep tech and physical infrastructure.   (0:00) Intro (2:41) China's Open Source Models Catch Up (7:42) Does Distillation Explain China's Rise? (13:54) The Geopolitical Risk of Chinese AI Models (20:28) Should the Government Restrict Open Models? (22:06) What Are the Labs Really Learning From You? (29:22) Future of Government Regulation (40:50) The OpenAI-Hugging Face Hack (47:59) Grok, Cursor, and the Value of Real Data (56:52) SSI and OpenRouter (1:02:52) Venture Capital's Return to Deep Tech (1:05:16) Quickfire Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Igor Babuschkin, co-founder of River AI and formerly a co-founder of xAI, joins to unpack a career that spans nearly every major AI lab: he led the StarCraft and AlphaCode work at DeepMind, joined OpenAI's reasoning team years before o1 shipped, and co-founded xAI, where he helped stand up the Colossus data center in roughly 120 days and reflects candidly on what it's actually like working with Elon Musk day to day, plus what the Cursor acquisition actually unlocked for Grok's coding models. He also discusses why he left xAI to start River AI, the three bets behind it, and why he's betting on local hardware, not just software, for personal AI. On the enterprise side, he tackles whether companies will actually train their own models or if it's just a cost play, and makes the case that proprietary labs like OpenAI and Anthropic are facing a real business squeeze. He's skeptical that stacking specialized RL domains generalizes the way pre-training scale did, and is candid about the uncomfortable reality that today's frontier open-weight models are almost entirely Chinese. He closes on what's actually needed to push model progress beyond coding into non-verifiable domains, and the broader implications of where AI is headed next.   (0:00) Intro (1:17) Writing Fiction on Where AI Is Headed (4:46) Cracking Agents Beyond Coding (10:29) Why Igor Left to Start River (12:22) River's Three Big Bets (18:06) Weights vs. Memory: The Personalization Debate (22:04) Should Enterprises Train Their Own Models? (25:10) Are Proprietary Labs Losing Their Edge? (32:16) The China Open-Source Problem (44:19) The Elon Call That Started xAI (50:18) Thoughts on Cursor Acquisition (52:16) What's Actually Bottlenecking AI (56:55) Humans, Machines, and Staying Relevant (1:01:29) Igor's Odds This All Goes Well Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Benedict Evans, one of tech's most widely-read analysts, joins Jacob Effron. The conversation centers on Benedict's core thesis that comparing AI's scale to past platform shifts (the internet, mobile, PCs) is analytically useless, and that the more productive move is studying how those previous technologies actually evolved economically to reason about where AI's value will accrue. He argues the one genuine difference this time is that we don't know AI's physical or scientific limits, unlike past shifts where the boundaries were at least knowable, and that this uncertainty is what fuels both AGI hype and doomerism without resolving anything. Benedict unpacks why capabilities remain jagged, meaning usage is jagged too, why coding became the first real enterprise use case thanks to scalable verification, and why most consumer and enterprise use cases still have to be invented by entrepreneurs rather than emerging spontaneously once models improve. He also lays out why foundation model labs may end up structurally like TSMC rather than Windows, valuable but bounded rather than owning the entire stack, walks through why automation has historically meant more work rather than less (using a hundred years of rising accountant headcount as evidence), and explains why industries like Uber and Airbnb, or Caterpillar and the internet, show just how unevenly this kind of technology actually lands. Throughout, he offers candid, historically grounded takes on OpenAI's product sprawl versus Anthropic's narrow coding bet, Apple's stumbled AI moment, and why most companies, unlike Silicon Valley, have far bigger priorities than AI on their minds.   (0:00) Intro (1:31) Is AI Bigger Than the Internet? (10:10) Barriers of Getting From Demos to Daily Use (20:15) Why Job Predictions Fail (25:52) Where's the Moat? (33:55) Will Models Eat the App Layer? (39:25) When Average Isn't Enough and Models Don't Work (45:58) Reflections on OpenAI (55:04) Consumer Usage Is Still Shallow (58:51) What's Required for More Enterprise Adoption (1:03:47) Opinion on Sora (1:06:27) Quickfire Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Dr. Jürgen Schmidhuber, a renowned scientist and AI researcher widely regarded as one of the pioneers in the field, originated key ideas behind today's transformers, LSTMs, and recursive self-improvement through his lab's work. He argues that true AGI remains bottlenecked by physical hardware, that today's AI data center investments are headed for a correction as open-source keeps pace with closed labs, and that the path to general intelligence runs through artificial curiosity and self-generated experimentation rather than internet data. He closes by reconsidering mainstream AI safety arguments and offers a sweeping vision of self-replicating robot societies eventually colonizing the solar system.   (0:00) Intro (1:24) How Close Is Superhuman AI? (2:27) Why ChatGPT Didn't Surprise Him (3:21) The Path to Recursive Self-Improvement (9:01) Will AI Takeoff Feel Sudden? (11:02) Intelligence Means Efficiency (12:32) Advice for Labs: Beyond Human-Biased Data (17:10) Artificial Curiosity and the Theory of Fun (21:33) When Do We Get the AI Scientist? (24:07) AI Chemistry, MOFs, and Carbon Capture (25:04) Robotics Reality Check (28:23) The Data Center Bet: Overbuilt? (31:48) Open Source vs. Closed Labs (34:25) Does Being First to RSI Create a Moat? (38:06) AI Safety and Alignment Skepticism (43:44) Quickfire Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

Six months after their last roundup, Jacob sits down with Ari Morcos (Datology AI CEO, former Meta AI researcher) and Rob Toews (Radical Ventures partner, Forbes AI columnist) to take stock of an AI landscape that has shifted dramatically: coding agents crossing the long-time-horizon threshold has turned engineers into managers of agents, near-frontier open weight AI looks like it may be disappearing as Meta and the Chinese labs pull back, and Anthropic's restrictions on its newly released Fable model have its biggest supporters questioning whether safety framing is masking competitive positioning. The conversation runs through the full state of the lab wars, including Rob doubling down on his Sam Altman ouster prediction and the Bret Taylor succession theory, why Google's structural advantages remain intact despite falling behind on coding, what xAI's Cursor acquisition is really for, and Ari's claim that compute constraints could push labs to suspend their APIs entirely. The back half digs into the physical bottlenecks underneath it all, from atom and x-ray lithography startups challenging ASML to H100 prices reversing their decline, before closing with predictions: recursive self-improvement is closer than it was six months ago but slower than the takeoff narratives suggest, robotics is nearing its GPT-3 moment, and Anthropic's next chapter may be life sciences.   (0:00) Intro (1:40) Coding Agents Cross a Threshold (3:29) Is Open-Weight AI in Retreat? (7:37) Cost Crunch & Scaffolding (12:13) The "Apps Are Cooked" Debate (16:37) Sam Altman Under Scrutiny (19:44) Anthropic's Fable Backlash (23:24) How Big a Step Change Is Fable? (26:50) What's Going On at Google? (33:20) Could the APIs Go Away? (34:11) Breaking the Semiconductor Bottleneck (35:42) Beyond EUV: Atom & X-Ray Lithography (37:23) Implications of a Compute Shortage (40:20) Do Alt Chips Actually Help? (43:43) SpaceX, xAI & the Cursor Acquisition (48:50) How Close Are We to RSI? (52:21) Quickfire Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.