Powering the AI Inference Wave with EPRI's Ben Sooter - Ep. 292

Enterprises are moving from AI pilots to full‑scale AI factories that turn data into trusted digital intelligence. Red Hat CTO Chris Wright and NVIDIA’s Justin Boitano unpack the "five‑layer cake" AI factory stack, from accelerated hardware and hybrid cloud infrastructure to models, agents, and production‑grade governance.

AI is reshaping electricity demand. What does increased demand, and the shape of that demand, mean for the electric grid? Ben Sooter, Director of R&D at EPRI joins the podcast to explain why most of an AI model’s lifetime energy use comes from inference rather than training, and how micro data centers located near underutilized substations can help deliver low‑latency AI services while strengthening grid resilience.

Alibaba.com president Kuo Zhang discusses how AI agents like Accio are reshaping global trade. He shares insights on automating complex B2B sourcing, compressing weeks of work into minutes, lowering barriers for solo entrepreneurs and SMEs, and what AI-native commerce will mean for the next decade.

Transit agencies are using AI and edge computing to keep bus lanes and bus stops clear — boosting on‑time performance, accessibility, and safety for riders. AC Transit CTO Ahsan Baig and Hayden AI CEO Marty Beard explain how bus‑mounted cameras and NVIDIA-powered edge AI automatically detect vehicles blocking bus lanes and stops, protect rider privacy by design, and are helping change driver behavior in the San Francisco Bay Area. Explore the next wave of AI innovation at NVIDIA GTC. Learn more.

How can AV teams stop drowning in petabytes of data and actually ship safer autonomy faster? Fortellix’s Rohan Bhasin and Voxel51’s Dan Gural explain how neural reconstruction, scenario-driven data curation, and NVIDIA-powered pipelines turn ordinary drive logs into high-fidelity simulations that close the last-mile gap in AV performance. GTC is the premier global AI conference. Learn more at nvidia.com/gtc

How are real‑time 4K holograms that look, sound, and respond like people changing fan engagement? Jia Li, co‑founder, president, and chief AI officer of LiveX AI, shares how human‑like AI agents bridge the digital and physical worlds—from greeting travelers at airports to guiding fans through fan zones and supporting wayfinding at major sports events. Join the world's premier AI conference: NVIDIA GTC

GiveDirectly president and CEO Nick Allardice explains how his team uses AI, mobile money, and satellite imagery to send cash directly to people living in poverty and crisis, often within days of a disaster. He describes how AI-powered tools help forecast floods in places like Nigeria, Bangladesh, and Mozambique, and how “anticipatory action” can get money to families before disaster strikes.  Learn more about the AI Podcast: ai-podcast.nvidia.com

Jason “Retailgeek” Goldberg, Chief Commerce Strategy Officer at Publicis Groupe, discusses how AI is  optimizes retail operations and is rewriting the consumer shopping experience. Learn why AI acceleration is able to reimagine the retail pipeline — from supply chain to personalized robot shoppers that could streamline the flow of goods starting at the warehouse and ending on your doorstep. More AI Podcast: ai-podcast.nvidia.com

Brandon Hootman, Vice President of Data and Artificial Intelligence at Caterpillar, joins the AI Podcast to discuss how the company uses NVIDIA’s AI Factory, Omniverse digital twins, and edge AI to streamline manufacturing, improve safety, and bring AI copilots into heavy equipment on real-world job sites. Browse the full AI Podcast catalog

Discover how mixture‑of‑experts (MoE) architecture is enabling smarter AI models without a proportional increase in the required compute and cost. Using vivid analogies and real-world examples, NVIDIA’s Ian Buck breaks down MoE models, their hidden complexities, and why extreme co-design across compute, networking, and software is essential to realizing their full potential.  Learn more: https://blogs.nvidia.com/blog/mixture-of-experts-frontier-models/