Ep. 034 - Engrams: How DeepSeek Offloads KV Cache to DRAM and SSD (Core Research) | Jordan Nanos, Cam Quilici, Alec Ibarra, Bryan Shan

Emergency pod. Hours after NVIDIA announced that Jensen Huang has resigned as CEO, with CFO Colette Kress named interim CEO, Jordan Nanos (@JordanNanos) sits down with Dylan Patel (@dylan522p) and ChinaTalk's Jordan Schneider (@jordanschnyc). They get into why the MATCH Act matters more than RASA, how much compute China should be allowed to rent abroad, the smuggling cases around NVIDIA's supply chain, China's push onto domestic accelerators, whether NVIDIA can execute without its founder, and who should run it next. Then it goes off the rails.Read the article: https://newsletter.semianalysis.com/p/nvidia-and-the-world-without-jensenChinaTalk: https://www.chinatalk.media0:00 Jensen Is Out2:04 MATCH Act vs RASA5:06 Why Not Ban Everything9:21 Will China Retaliate15:54 The Smuggling Evidence21:09 China's Domestic Chips23:08 Flip the Table29:03 Open Models and Talent31:10 NVIDIA Without Jensen41:46 Who Runs NVIDIA Next45:48 Closing Thoughts47:56 The Nationalization DebateThe indictments discussed are allegations; defendants are presumed innocent. Opinions are the speakers' own. Not financial advice.

Recorded Election Day morning, before any results: SemiAnalysis’s model has about 62 GW of US datacenter additions in 2028, most not yet under construction and sitting in states electing a governor. Texas alone is close to 30%.Jordan Nanos (@JordanNanos) sits down with Maya Barkin (@mayabarkin), Wayne Ma (@waynema) and Jeremie Eliahou Ontiveros (@JeremieEO) on why 2027 is largely locked in, why the backlash crosses party lines, and how developers can win over their neighbors.Since recording: the AP called Texas for Gina Hinojosa, who would pause datacenter permitting, and Georgia for Keisha Lance Bottoms, and Democrats took a 3-2 majority on Georgia’s Public Service Commission, which sets power rates. (As of Nov 4, 2:35 PM ET.)Read the article: https://newsletter.semianalysis.com/p/the-datacenter-backlash-goes-bipartisan0:00 Intro0:54 The 2028 Buildout5:32 Pause vs. Conditions10:03 Bipartisan Backlash14:37 Grid Risk16:30 Tax Incentives19:51 Texas24:34 Risks to 202730:23 Georgia32:31 Federal Outlook35:20 Winning Over Neighbors42:04 What We're Watching

SemiAnalysis just launched GDPx, a benchmark built with 14 specialized data partners across 11 domains where frontier intelligence earns the highest returns. Even the best model scores just 12%. Jordan Nanos (@JordanNanos) sits down with Max Kan (@maxkan) and Andrew Megalaa (@AndrewMegalaa) to explain why most popular benchmarks are broken, the difference between difficult and unfair tasks, how a real private equity deal room became a finance environment, why binary grading still matters, and what the leaderboard says about Opus 5.5, GPT-6 Astra and Gemini 4 Carbon.Explore GDPx: https://gdpx.semianalysis.comRead the article: https://newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilitiesWant your model evaluated or to partner on a future version? [email protected]:00 Cold Open0:10 Launch Week Chaos4:02 Broken Benchmarks5:15 Difficult vs. Unfair7:33 Building GDPx9:25 Frontier Domains11:02 The 11 Domains14:12 Domain Experts15:00 Models vs. Experts19:36 Model Behavior20:44 The Leaderboard22:33 Cyber and Safety24:32 Frontier vs. Day-to-Day27:13 The Finance Task31:27 Binary vs. Partial Grading36:48 AI and Jobs42:43 Quick Takes43:44 Google's Comeback47:25 Public vs. Private Tasks49:52 What's Next

Anthropic filed its S-1: revenue per dollar of compute is up 5.2x since Q1 2025, a customer who spent $1 last year now spends $9, and incremental gross margins are running near 90%. Jordan Nanos (@JordanNanos), Doug O'Laughlin (@fabknowledge), Jeremie Eliahou Ontiveros (@JeremieEO) and Joey Brookhart (@SaasquatchC) break down profitability, the half trillion in compute commitments, customer concentration, the 2027 compute roadmap, and whether OpenAI can keep up.0:00 Cold Open0:53 Profitability2:10 The Datacenter View4:14 Compute Commitments6:28 Incremental Margins8:19 Customer Concentration12:00 Top Customers16:53 New Funding18:54 Underpromise, Overdeliver20:25 Elon and Neoclouds25:14 Compute Efficiency29:40 Can Anyone Catch Up?33:26 Surprises and Runway37:47 Market Reaction39:02 Holiday Usage42:56 Is OpenAI Cooked?48:44 Government Revenue50:45 Wrap-Up

Every hybrid bonder on the market today brings the die to the wafer. ASML's patents bring the wafer to the die.Sherry Zhang and Howie join Jordan Nanos (@jordananos) to dig into the patents behind ASML's die-to-wafer hybrid bonder, why a track-based design could beat Besi on alignment and particle control, and why HBM won't adopt die-to-wafer hybrid bonding before 2029. The team also looks at how China is using hybrid bonding to keep scaling past its lithography limits, and what larger photomasks mean for High-NA EUV.Chapters:0:00 The Patent Tip-Off1:53 Hybrid Bonding 1014:06 Die-to-Wafer vs Wafer-to-Wafer5:43 Back End vs Front End8:05 Why HBM Waits Until 202913:45 Everything Comes to the Die18:26 Alignment and Cleanliness22:48 Bonding Instead of Shrinking25:49 China's Toolmakers Step Up28:25 Bigger Masks for High-NA33:29 Two Tool Stacks?Read the full article: ASML Is Building a Hybrid Bonding Tool!https://newsletter.semianalysis.com/p/asml-is-building-a-hybrid-bonding

Brad Gerstner put our 43 gigawatt forecast for 2027 on All-In, then again in his BG2 sit-down with Jensen, and called it too aggressive. The consensus counter sits near 25 GW, built on moratorium headlines and grid connection fears. Jordan Nanos (@JordanNanos) sits down with Jeremie Ontiveros (@JeremieEO) and Reyk Knuhtsen (@robotknower), who have been tracking the same capacity curve since March 2024, and it has barely moved across every model revision. So they aren't cutting it. They're raising it.Read the article: https://newsletter.semianalysis.com/p/datacenters-2027-the-year-of-acceleration0:00 Cold Open1:17 The 2027 Forecast Debate2:21 Model Track Record5:27 Where the Pushback Comes From8:16 Delays and Haircuts12:31 The Top 40 Sites15:23 Leading Indicators20:40 Modular Builds25:41 Chips vs Datacenters27:53 Responding to Brad Gerstner

GLM 5.3 only attends to the tokens that matter. It still has to keep every one of them.Kimbo Chen (@kimbochen) and Wenyao Gao (@gao_wenyao) join Jordan Nanos (@JordanNanos) to dig into AMD catching NVIDIA on InferenceX, how post-training alone pushed an open-weight model within reach of Mythos, and whether Z.ai quietly tuned it for Chinese chips. The team also looks at how GLM 5.3 became a fallback when Claude and GPT refuse security work.Chapters:0:00 Cold Open0:37 Sparse Attention 1014:46 HiCache and HiSparse10:13 AMD Catches NVIDIA12:54 Capacity vs Bandwidth18:44 The Frankenstein Flash20:09 Mythos-Class From China?22:52 Post-Training Pipeline32:32 Chinese Chips, Day One39:54 GLM 5.4 and BeyondRead the full article: How GLM 5.3 Sparse Attention Affects HBM Memory Usagehttps://newsletter.semianalysis.com/p/sparse-savings-persistent-demand-inside-glm53

H100 rental prices went up, on a GPU that should be sliding down its depreciation curve. That curve is what lenders underwrite. Daniel Nishball (@dnishball) kicks off our new Compute Capital Markets group by following the money through Nvidia's $497B backstops, ClusterMAX as a credit signal, and whether each token is profitable. Jordan Nanos (@JordanNanos) brings the rental data. AI debt is about to pass auto loans.0:00 Opening1:48 Nvidia's Backstop Universe6:22 The $11T Funding Wall17:25 Balance Sheet as a Service26:27 ClusterMAX as Credit Risk31:51 Why H100 Prices Rose43:09 Token Profitability58:05 You Can't Go Back1:07:25 AI Debt vs Auto Loans1:11:14 Inside Nvidia's Backstops

Pay Anthropic $200 a month and you can pull $12,000 of Opus 5.5 tokens at API pricing. Max Kan (@maxkan) and Andrew Megalaa bought every plan, OpenAI, Meta, xAI, MiniMax, Moonshot, Z.ai, Cursor, and measured every token type to find the ceiling. The catch isn't subsidy. Anthropic still books 50 to 60% gross margins on these plans, which means the API itself is the price gouge.0:00 Cold Open1:03 Subsidies and Credits5:23 Token Types and Cache14:08 Model Value Charts20:51 Test Methodology26:54 Other Plans31:26 Utilization and Limits36:24 Token Efficiency41:24 Open Models44:22 Cyber and CaptureRead More: https://newsletter.semianalysis.com/p/anthropic-subscriptions-offer-5x

Jordan (@JordanNanos) and Abhilash (@AbhilashJain17) discuss the clients SemiAnalysis consulting serves, custom projects like the inference simulator, how client work turns into research products, neocloud due diligence, and why SLAs decide who gets funded. They also discuss Feynman specs, de-specced memory, and a 20-trillion-parameter OpenAI model. 0:00 Intro and Overview2:27 Client Types5:24 Custom Projects9:47 Strategy Work12:27 Consulting to Products14:33 Neocloud Due Diligence18:40 SLAs and Insurance22:32 Risk Assessments26:45 Build vs Buy29:19 Favorites and Outlook