Zach Wilson - Data Engineering in 2026, Traveling, and more - Freestyle Fridays - May 8, 2026

It's been a few months on the road, bouncing through San Francisco a bunch, across Asia and Europe, and a quick stop in Detroit. In this audio-only Freestyle Friday I unpack what I've been seeing out there. If I had to pick one word for the mood worldwide, it's uncertainty: energy and supply shocks rippling out of the Middle East, fuel and resource shortages, flights getting canceled with no notice, and AI scrambling the playbook for vendors, practitioners, and leaders alike.I get into why so many data tooling companies are quietly having existential conversations, how Atlan tore its product down to rebuild AI-native (a full conversation with Prukalpa is coming next week), and a fun experiment I shipped this week with DuckDB Quack.I also dig into the split I keep seeing: senior practitioners getting superpowers while juniors face a brutal job market, leaders being asked to do far more with less, and why I think the industrial-age org chart is finally on its way out.Plus some personal updates: the new book is now targeting late July and a companion course is on the way.Finally, I'm mixing audio and video formats going forward (Freestyle Friday will probably be mostly audio), the Practical Data Community newsletter is live, and there's a Salt Lake City conference brewing for late January. Lots in the hopper...------------------This episode is sponsored by Revefi, who gives you full cost and performance visibility for Snowflake by warehouse, user, and workload. One team cut Snowflake costs ~50% across 711 warehouses in under 48 hours. Book a demo at revefi.com/demo.------------Timestamps0:00 — Intro & travel recap — Sets the stage: months of globe-trotting across Asia, Europe, and the US1:10 — Global uncertainty & resource scarcity — Fuel/water shortages in Southeast Asia, flight cancellations in Europe, ripple effects of geopolitical tensions5:30 — AI dominates every conversation — The #1 topic at conferences worldwide; vendors facing existential questions and forced to rethink everything (Atlan pivot, DuckDB agent idea)10:14 — AI's impact on workers at every level — Senior practitioners gaining superpowers, juniors worried about jobs, leaders expected to do more with less17:51 — Key takeaway: everyone feels behind — Even top AI insiders are uncertain; give yourself grace, upskill, and consider building something for yourself20:38 — Announcements — Book drops July 27th, course coming, Practical Data Community Newsletter live, fall travel schedule (London, Paris, possible Salt Lake City conference)

If you're a consultant and you're not using AI agents yet, your competitors are. No surprise, but they're delivering faster, cheaper, and better than ever.Chris Tabb, founder of LEIT Data, joins me live at Confluent Current London 2026 to talk honestly about how AI agents are reshaping the consultancy model, from billing structures and team rollouts, to building internal tribal knowledge and outpacing firms that are still staffing up the old way.Timestamps:0:33 — How Chris is Going Agentic1:56 — Token Maxing Leaderboards5:26 — AI Agents: Year-Over-Year7:08 — Tagile: Agentic Development9:00 — AI in Consultancy17:22 — Prompt Management & Context Quality

Everyone in tech is telling you to go faster. Eric stepped away from his role to do the opposite.In this conversation, we get into why so many people feel like they're falling behind in AI, and why that feeling is mostly manufactured. Eric makes the case that we're miscalibrated: assuming what's true for the 0.1% (the SF AI inner circle) is true for the 10%, when by definition almost no one is keeping up with that group. We talk about why judgment, not throughput, is the real bottleneck right now, why most AI products feel boring even as code output explodes, what the "flatten the org" experiments are actually measuring (spoiler: yesterday's stock price), and why people are leaving corporate roles at a rate that's hard to ignore.We also get into the parts nobody wants to say out loud: layoffs by email, the gap between who people are in private vs. on LinkedIn, the ghost routines after you leave a job, and what walking around San Francisco actually feels like when every billboard is AI and every person you pass looks depleted.If you've felt the FOMO and wondered whether the problem is you or the framing, this one's for you.Eric Weber is a data and product leader formerly at Grammarly, Yelp, LinkedIn, and Stitch Fix.

In this episode, Tristan Handy and I sit down to unpack a massive shift coming to the data industry: over the next 12 months, the primary consumers of data won't be humans. They will be AI agents. We dive deep into what this means for data infrastructure, compute costs, and the tools we use every day. We also talk about processing high-volume agent queries, building "context stores", and why the industry shouldn't just build "horses with wheels" when designing agentic data engineers. We also take a fun detour comparing the current AI landscape to the early days of dial-up modems and Mosaic browsers , and discuss why stepping away from the screen and going old-school might be the ultimate productivity hack.

NOTE - Sorry for the edits in this video. I used Descript to edit out the umms and uhhs, and it was a bit too aggressive. Will make it less jarring in future videos. Thanks.Freestyle Friday, May 15, 2026Walking around Salt Lake City and unpacking the April 2026 data modeling survey results (334 respondents). Across three surveys now: January's State of Data Engineering (1,100), March's AI usage poll (193), and April's data modeling deep-dive. Not surprisingly, the same two pain points keep surfacing: time pressure and lack of clear ownership.90% of respondents have a data modeling pain point. When asked what would actually help, only 4.8% wanted better tools. Training, business requirements, time, and ownership crushed tooling in the rankings. Will AI improve things or make them worse? Time will tell...Also covered:Why physical data modeling has become the default (and why that's a problem)Data modeling vs. schema design - they're not the same thingSemantic layers (yay or nay?), Lloyd Tabb, and MalloyConway's Law, Reis's Law, and what changes when org charts get flattened by AIWhy leadership is under more pressure than everThe June half-year survey is coming🎙️ SPONSORSFivetran - stop cobbling pipelines together. Set it, forget it, scale as you grow.→ https://fivetran.comRevify - surprise Snowflake bills? One customer cut theirs 50% in 48 hours.→ https://revify.com/demo

Are AI agents silently draining your cloud data budget? With the rise of consumption-based pricing and autonomous AI queries, data teams are facing a perfect storm of skyrocketing costs and operational chaos. In this episode, I sit down with Sanjay Agrawal, CEO and Co-founder of Revefi, to discuss the intersection of data engineering, cloud warehouse optimization, and FinOps in the age of AI.We chat about how legacy on-prem habits are bankrupting modern data platforms, why query optimization is more about ROI than just speed, and how AI agents are changing the landscape of data consumption. Sanjay shares his deep expertise from building world-class databases at Microsoft and ThoughtSpot, revealing how to automate cost management and performance tuning for Snowflake, Databricks, and BigQuery.Key Topics:The evolution of cloud data warehouse pricing and why it breaks traditional budgets.How AI agents are causing massive, unpredictable spikes in compute spend.Real-world horror stories of ""lift and shift"" cloud migrations.Why database benchmarks focus on speed but ignore the actual ROI of data.The future of open table formats (Iceberg) and multi-engine routing.

Zach Wilson and I happen to be in Stockholm, Sweden, this evening. In this Freestyle Friday chat, we talk about what it takes to be a data engineer in 2026 and much more.

Josh Wills has spent 25 years writing data pipelines, with a career spanning Cloudera, as Director of Data Engineering at Slack, on the dbt DuckDB adapter, and now training foundation models at Datology AI. He uses coding agents every day. And he keeps running into the same wall: the agents jump to conclusions, fix the wrong thing, and ship pipelines no one understands.In this conversation, we unpack why AI agents struggle with the messiest, highest-stakes parts of data work, and what it means for the engineers managing them.We get into:- Big Data is back- Why AI agents jump to conclusions on benchmarks and complex bottlenecks- The $200K vibe-coded pipeline problem nobody wants to talk about- Why there's no training data for the gnarly enterprise pipelines that actually power businesses- "We're all managers now" - managing unreliable agents like managing unreliable people- Wicked problems and the limits of intelligence- Why politics is the last human endeavor to fall to LLMs (the data is never written down)- Whether classical ML still has a place (yes)- What Josh would tell a new grad starting in data today

Stop Tokenmaxxing and step off the AI hamster wheel. Welcome to another Freestyle Friday! What's the overwhelming vibe in the AI zeitgeist? "If you aren't maxing out AI every second, you're going to be left behind." Therefore, Tokenmaxxing is the way, right?I strongly disagree. We're burning ourselves out with fake productivity and a graveyard of abandoned AI-generated projects.In this episode, I talk about my new minimalist travel setup, why I'm purposefully trying to minimize my AI usage for deep cognitive work, and what skills will actually get you left behind (hint: it's not missing the latest model release).Solve real problems. Focus on the fundamentals. Take a walk. You're going to be fine.

Shinji Kim, founder of SelectStar (acquired by Snowflake in December), joins the show to discuss the deal, the integration into Snowflake's Horizon catalog, and where data cataloging is actually headed.We get into the weeds on a claim Shinji makes early: in a few years, we may stop calling these things "data catalogs" at all. The category is evolving into an AI context layer, a living surface that combines metadata, semantic models, business glossaries, and ontologies, continuously updated by both humans and agents. Shinji walks through how SelectStar built toward this with semantic model management, MCP server support, and an AI agent that started serving data analysts and eventually answered business users' questions directly.We also dig into where data catalog implementations go wrong (spoiler: it's almost always adoption, not tooling), why marketing teams are an underrated ETL persona, and what it actually took to get acquired by Snowflake after three years as a premier partner.Plus: if Shinji were starting SelectStar today, what would she do differently? We talk about distribution in the AI era and how the startup playbook is mutating.Connect with Shinji:LinkedIn — https://www.linkedin.com/in/shinjikim/