Success without Dignity? Nathan finds Hope Amidst Chaos, from The Intelligence Horizon Podcast

This special cross-post from The Intelligence Horizon features Nathan Labenz in a wide-ranging conversation on compressed AI timelines, expert disagreement, and why he believes the singularity is near. They discuss interpretability, RL scaling, and the balance between extraordinary upside, like curing major diseases, and serious existential risks. Nathan explains his evolving p(doom), why he’s slightly more optimistic about robustly good AI, and how defense-in-depth strategies might keep society on track. The episode also explores US-China rivalry, AI governance, and why human cooperation may matter more than technical control alone. Google: Keep up with AI research on the go with NotebookLM, Google's steerable research and thinking partner. Try it at https://notebooklm.google.com/. Sponsors: Tasklet: Build your own Cognitive Revolution monitoring agent in one click.Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (03:27) Special Sponsor (05:12) Opening and AGI framing (12:08) Scaling RL and paradigms (Part 1) (21:31) Sponsors: Tasklet | VCX (24:24) Scaling RL and paradigms (Part 2) (28:56) Verifiability and long horizons (41:13) LLMs and world models (Part 1) (41:19) Sponsor: Claude (43:32) LLMs and world models (Part 2) (54:17) Energy, hardware, and chips (01:00:42) Alignment risks and bottlenecks (01:10:18) AI values and agency (01:20:31) Defense in depth alignment (01:30:48) US-China AI cooperation (01:41:05) Episode Outro (01:45:42) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Vijoy Pandey of Outshift by Cisco lays out his vision for an “Internet of Cognition,” where AI agents can share context, build reputation, and collaborate safely at scale. He offers a useful mental model for superintelligence: progress has to scale in two directions — up, through better individual models, and out, through networks of agents and humans thinking together. The conversation explores how distributed, protocol-driven agent systems could give enterprises fine-grained permissions, auditability, and controlled interfaces, in contrast to today’s centralized frontier models. Vijoy also walks through Cisco’s internal CAIPE system of 20 cooperating agents, the open-source AGNTCY project, and a live multi-agent healthcare demo spanning diagnostics, insurance, pharmacy, and scheduling. LINKS: AGNTCY Project Open source multi-agent infrastructure under Linux Foundation governance. Covers discovery, identity, communication, observability. Vijoy walks through the architecture at [00:34:57] and [00:41:17]. Scaling Out Superintelligence Whitepaper The technical whitepaper detailing the Internet of Cognition architecture, three-layer stack, and cognition state protocols. Referenced at [01:25:40]. Internet of Cognition Interactive Demo Clickable walkthrough showing per-agent activity, intent, context, and collective reasoning across a multi-agent SRE system. Vijoy demos at [01:26:20]. CAIPE Project (GitHub)  Cloud Native AI Platform Engineer. Multi-agent system with participation from Adobe, AWS, Cisco, Nike. 20 agents, 100+ tool calls, 10+ workflows. Referenced at [00:11:52]. Sponsors: Tasklet: Build your own Cognitive Revolution monitoring agent in one click.Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (04:16) Cisco and networking foundations (13:34) Jarvis and ASI vision (Part 1) (18:16) Sponsors: Tasklet | VCX (21:09) Jarvis and ASI vision (Part 2) (Part 1) (31:46) Sponsor: Claude (33:59) Jarvis and ASI vision (Part 2) (Part 2) (34:00) Practical multi-agent examples (50:02) Multi-agent plumbing architecture (01:01:44) Agent identity and TBAC (01:15:23) Internet of cognition fabric (01:21:48) Emergent agents and safety (01:36:52) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Karan Vaidya, CTO of Composio, explains how their “smart tool” platform lets AI agents access over 50,000 tools across 1,000+ apps through a single interface. He details how Composio handles tool discovery, authentication, sandboxes, and logging, and how an AI-powered feedback loop continuously improves tools in real time. The conversation explores avoiding model lock-in through robust skills and instructions, translating capabilities across model providers, and why the best agent use cases look more like full jobs than isolated tasks. Google: Try Google's latest and greatest model, Gemini 3.1 Pro, in AI Studio (https://aistudio.google.com/) or the Gemini app. Sponsors: Tasklet: Tasklet: Build your own Cognitive Revolution monitoring agent in one click.Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (03:38) Special Sponsor (05:10) Composio overview and harness (10:20) Users, trust, security (19:45) Sandboxes and execution (Part 1) (19:53) Sponsors: Tasklet | VCX (22:46) Sandboxes and execution (Part 2) (28:07) Smart MCPs and skills (Part 1) (34:25) Sponsor: Claude (36:38) Smart MCPs and skills (Part 2) (44:10) Context, access, upgrades (54:05) Skills and model lock-in (01:03:51) Memory and agent tools (01:09:21) AI and SaaS disruption (01:20:20) Agents, costs, labor (01:31:18) Monetization and interfaces (01:36:13) Episode Outro (01:39:56) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Zvi Mowshowitz returns to survey the current AI landscape, from recursive self-improvement and the shift from the “beginning” to the “middle” of the AI story to what true AI end-game would look like. He and Nathan dig into AI-driven job loss, real-world productivity impacts, and the ethics of trying to escape a “permanent underclass.” They assess today’s AI live players, why Anthropic may be slightly ahead, and whether Chinese, xAI, or Meta can catch up. The conversation closes with Anthropic’s Responsible Scaling Policy, p(doom), AI safety options, and how they each use AI in their own work. Sponsors: Tasklet: Tasklet: Build your own Cognitive Revolution monitoring agent in one click.Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com CHAPTERS: (00:00) About the Episode (02:25) Entering the middle game (09:08) AI layoffs and jobs (Part 1) (15:50) Sponsors: Tasklet | VCX (18:43) AI layoffs and jobs (Part 2) (18:44) AI growth and elites (27:00) Defining the AI endgame (36:09) Live players and laggards (45:38) China, compute, and distillation (56:03) Meta, Musk, and strategy (01:06:41) Google's faltering AI strategy (01:22:25) Anthropic's scaling policy shift (01:36:29) Anthropic and domestic surveillance (01:57:29) Courts, power, and Anthropic (02:18:50) Model fatigue and productivity (02:34:53) Alignment basins and doom (02:47:24) Slowing AI and activism (03:05:37) Forbidden techniques and choices (03:22:31) Episode Outro (03:26:31) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

This special AI Scouting Report episode from the Law & Artificial Intelligence Certificate Program surveys the current AI landscape for legal professionals. Nathan Labenz walks through the “Good, Bad, and Weird” of frontier models, from using AI to navigate his son’s cancer treatment to emerging forms of deception and reward hacking. He highlights how new systems are pushing the boundaries of math, physics, and legal performance while raising serious safety and governance questions. Listeners will come away with a fast-paced, source-rich overview of where AI is today and the strange future it’s steering us toward. LINKS: Google: Try Google's latest and greatest model, Gemini 3.1 Pro, in AI Studio or the Gemini app.Presentation Link Sponsors: Tasklet: Tasklet: Build your own Cognitive Revolution monitoring agent in one click.Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (03:04) Gemini's long context window (05:23) Comprehensive AI overview (Part 1) (16:01) Sponsors: Tasklet | VCX (18:53) Comprehensive AI overview (Part 2) (Part 1) (34:39) Sponsor: Claude (36:51) Comprehensive AI overview (Part 2) (Part 2) (59:43) Reward and sentience (01:04:14) Regulating bad actors (01:09:30) Corporate AI strategies (01:14:59) Episode Outro PRODUCED BY: https://aipodcast.ing

Jassi Pannu, Assistant Professor at Johns Hopkins, explains how rapidly advancing AI is transforming biological research and raising the risk of engineered pandemics. They map today’s biosecurity landscape, from pathogen detection and DNA sequencing to vaccine development, and examine how frontier models can already troubleshoot lab work and bypass data safeguards. The conversation introduces a proposed Biosecurity Data Level framework to restrict only the most dangerous functional biological data while preserving open science. They close with a broader defense-in-depth strategy—Delay, Deter, Detect, Defend—including DNA synthesis screening, global pathogen surveillance, and practical tools like Far UV sterilization. LINKS: Jassi Pannu: https://x.com/JassiPannuMD Science article that prompted this conversation: https://www.science.org/doi/10.1126/science.aeb2689 Sponsors: VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Framer: Framer is an enterprise-grade website builder that lets business teams design, launch, and optimize their.com with AI-powered wireframing, real-time collaboration, and built-in analytics. Start building for free and get 30% off a Framer Pro annual plan at https://framer.com/cognitive Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr Tasklet: Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai CHAPTERS: (00:00) About the Episode (05:59) From outbreak to vaccine (17:08) Threat actors and data (Part 1) (21:23) Sponsors: VCX | Framer (23:53) Threat actors and data (Part 2) (31:05) Gain-of-function research risks (Part 1) (37:39) Sponsors: Claude | Tasklet (41:03) Gain-of-function research risks (Part 2) (48:05) AI models in biology (01:00:51) Dangerous AI capabilities (01:07:59) Biosecurity data level framework (01:18:58) Policy, governance, and infrastructure (01:28:53) Defense in depth vision (01:40:43) Episode Outro (01:45:02) Outro PRODUCED BY: https://aipodcast.ing

Jesse Genet shares how she built a team of AI agents to transform homeschooling, family life, and personal productivity without a software background. She explains how agents like an AI chief of staff, curriculum planner, and content creator help design personalized lessons, analyze kids’ learning, manage educational toys, and even run TikTok. The conversation covers practical delegation workflows, guardrails and trust, and why she treats AIs like employees with onboarding and clear roles. Jesse also explores local models, privacy, and how AI in the home could reshape future work and family life. Use the Granola Recipe Nathan relies on to identify blind spots across conversations, AI research, and decisions: Sponsors: VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr Serval: Serval uses AI-powered automations to cut IT help desk tickets by more than 50%, freeing your team from repetitive tasks like password resets and onboarding. Book your free pilot and guarantee 50% help desk automation by week 4 at https://serval.com/cognitive Tasklet: Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai CHAPTERS: (00:00) About the Episode (04:57) Homeschooling context and AI (15:55) Building an AI team (Part 1) (19:51) Sponsors: VCX | Claude (23:18) Building an AI team (Part 2) (31:03) Onboarding agents like employees (Part 1) (38:12) Sponsors: Serval | Tasklet (40:31) Onboarding agents like employees (Part 2) (40:57) Context, models, and privacy (48:47) AI intimacy and rights (56:19) Coordinating agents in Slack (01:02:19) Designing an agent superapp (01:08:35) Agent trust and kids (01:17:57) Voice interfaces for families (01:29:51) Curated screens and automations (01:40:28) Sharing setups and software (01:48:43) Local sovereignty and kid devices (01:59:26) Work, disruption, and play (02:04:58) Episode Outro (02:07:45) Outro PRODUCED BY: https://aipodcast.ing

Dan Balsam and Tom McGrath from Goodfire return to explore the frontier of mechanistic interpretability and their new research pillar, Intentional Design. They explain the shift from sparse autoencoders to understanding geometric structure in latent spaces, and share a proof-of-concept method for reducing hallucinations using probes and RL. The conversation tackles concerns about reward hacking, principles for shaping the loss landscape instead of fighting backprop, and what this means for aligning powerful models. They also discuss recent Goodfire results on Alzheimer’s prediction, disentangling memorization vs reasoning weights, and how they balance commercial growth with a public benefit mission. Nathan uses Granola to uncover blind spots in conversations and AI research. Try it at granola.ai/tcr with code TCR — and if you’re already using it, test his blind spot recipe here: https://bit.ly/granolablindspot LINKS: Detecting PII for Rakuten Interpretability for Alzheimer's biomarker detection You and Your Research Agent Adversarial examples and superposition Discovering rare behaviors with model diff Priors in time for interpretability Belief dynamics in in-context learning Mixing mechanisms in language models Sparse autoencoder scaling with manifolds Sponsors: VCX: VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr Serval: Serval uses AI-powered automations to cut IT help desk tickets by more than 50%, freeing your team from repetitive tasks like password resets and onboarding. Book your free pilot and guarantee 50% help desk automation by week 4 at https://serval.com/cognitive Tasklet: Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai PRODUCED BY: https://aipodcast.ing

Geoffrey Irving, Chief Scientist at the UK AI Security Institute, explains why our theoretical understanding of machine learning remains fragile even as models surpass experts on critical security tasks. He details AISI’s work on frontier model evaluations, red teaming, and threat modeling across biosecurity, cybersecurity, and loss-of-control risks. The conversation explores reward hacking, eval awareness, and why current safety techniques may struggle to deliver high reliability. Listeners will also hear how AISI is funding foundational research to build stronger guarantees for AI safety. Nathan uses Granola to uncover blind spots in conversations and AI research. Try it at ⁠granola.ai/tcr⁠ with code TCR — and if you’re already using it, test his blind spot recipe here: ⁠https://bit.ly/granolablindspot⁠ Sponsors: Serval: Serval uses AI-powered automations to cut IT help desk tickets by more than 50%, freeing your team from repetitive tasks like password resets and onboarding. Book your free pilot and guarantee 50% help desk automation by week 4 at https://serval.com/cognitive Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr Tasklet: Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai CHAPTERS: (00:00) About the Episode (04:09) From physics to ML (08:52) AGI uncertainty and threats (Part 1) (18:08) Sponsors: Serval | Claude (21:29) AGI uncertainty and threats (Part 2) (27:35) Control, autonomy, alignment (Part 1) (34:02) Sponsor: Tasklet (35:14) Control, autonomy, alignment (Part 2) (38:44) Inside the UK AC (51:02) Evaluations and jailbreaking (01:01:17) Emerging capabilities and misuse (01:14:20) Agents and reward hacking (01:26:09) Theoretical alignment agenda (01:38:39) Debate and formal methods (01:51:19) Limits of formalization (02:02:27) Future risks and governance (02:16:23) Episode Outro (02:18:58) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Karan Singhal, Head of Health AI at OpenAI, explains how ChatGPT Health is achieving attending-physician-level performance and already serving hundreds of millions of users. He details how OpenAI works with over 250 doctors, built the 49,000-criteria HealthBench evaluation, and ran one of the first randomized trials of AI copilots in clinical care. The conversation explores privacy and safety safeguards, medical multimodality, N-of-1 treatment plans, and how AI could become a standard part of global medical practice. Nathan uses Granola to uncover blind spots in conversations and AI research. Try it at ⁠granola.ai/tcr⁠ with code TCR — and if you’re already using it, test his blind spot recipe here: ⁠https://bit.ly/granolablindspot⁠ LINKS: modeling human wellness Sponsors: Claude: Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr Serval: Serval uses AI-powered automations to cut IT help desk tickets by more than 50%, freeing your team from repetitive tasks like password resets and onboarding. Book your free pilot and guarantee 50% help desk automation by week 4 at https://serval.com/cognitive Framer: Framer is an enterprise-grade website builder that lets business teams design, launch, and optimize their.com with AI-powered wireframing, real-time collaboration, and built-in analytics. Start building for free and get 30% off a Framer Pro annual plan at https://framer.com/cognitive Tasklet: Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai CHAPTERS: (00:00) About the Episode (06:11) Cancer story and mission (11:46) Designing safe health AI (Part 1) (17:49) Sponsors: Claude | Serval (21:09) Designing safe health AI (Part 2) (26:48) Uncertainty, HealthBench and robustness (Part 1) (30:23) Sponsors: Framer | Tasklet (32:50) Uncertainty, HealthBench and robustness (Part 2) (38:11) Chain-of-thought and evaluation (46:49) Real-world performance and frontiers (55:35) Multimodal data and science (01:05:36) Personalization, privacy and monitoring (01:15:47) Models, data and incentives (01:29:31) Doctor adoption and workflows (01:38:13) Scalable oversight and alignment (01:51:06) Move 37 and future (02:00:50) Episode Outro (02:03:06) Outro PRODUCED BY: https://aipodcast.ing