
How to Pick the Right Data Warehouse for Your Workload, Backed by a 7-Engine Benchmark
Seven cloud data warehouses, one open benchmark, and no single winner.
Your warehouse choice probably came down to what someone used at their last job, plus a vendor bake-off with the outcome decided before it started. Then the bill arrived.
James Winegar and Ross Katz from CorrDyn, who ran the benchmark, alongside Dave Yaffe, Co-Founder of Estuary break down what happened when they loaded the same data into Databricks, BigQuery, Redshift, Microsoft Fabric, ClickHouse, MotherDuck and Snowflake. You'll come away with a sharper way to shortlist engines, size compute and challenge vendor claims before you commit budget to your data stack.
The conversation covers warehouse sizing, concurrency, idle time billing, open table formats like Iceberg, and how AI agents change query load on your data infrastructure. It's built for heads of data, CTOs and data engineering leads weighing a warehouse decision. If you want one tool crowned the winner, this benchmark won't give you that.
Key Takeaways
- Under 10GB, pick the cheapest engine you can run. At 100GB and 1TB, the speed-versus-cost curve flips, and a bigger warehouse can cost you less.
- "Useful concurrency" combines team size, data volume and a latency target into one sizing number. AI agents can make a ten-person team query like a thousand people.
- Redshift and Fabric beat their reputations in testing, with caveats that bite if you run 24/7 or juggle many workloads.
- A second vendor you've tested on your own workload is your best leverage when a three-year contract lands on your desk.
Chapter Markers
00:00 Why build a warehouse benchmark at all
02:17 Why Estuary backed the project
03:33 Data size, concurrency and cost dimensions
04:44 How teams really pick warehouses today
06:02 Why vendor-run benchmarks fall short
10:13 Landing 2.2TB into seven engines
14:11 Query validation for apples-to-apples results
15:21 Sizing compute: speed versus cost curves
19:43 Scan aggregates, ClickHouse and MotherDuck
22:10 Defining useful concurrency
27:34 How idle time drives your bill
32:36 Reputations confirmed and contradicted
39:15 Home field advantage caveats
42:14 Open table formats and managed Iceberg
47:29 Agentic analytics and warehouse load
53:10 AI agents inside the data pipeline
58:06 No clear winner across the board
1:01:35 Why autoscaling was switched off
1:03:21 How to read any vendor benchmark
1:05:43 Before you sign a three-year contract
1:08:57 What could change in a rerun
Useful Links & Resources
- Data warehouse buyer's guide and companion app (built on this benchmark)
- Benchmarks referenced: TPC-DS and ClickBench
- Apache Iceberg (open table format)
- CorrDyn: https://corrdyn.com
Connect With the Show
- CorrDyn on LinkedIn: https://www.linkedin.com/company/corrdyn/
- Ross Katz on LinkedIn: https://www.linkedin.com/in/b-ross-katz/
- James Winegar on LinkedIn: https://www.linkedin.com/in/james-winegar/
- David Yaffe on LinkedIn: https://www.linkedin.com/in/davidyaffe/
- Website: https://corrdyn.com
Which warehouse reputation does your own experience contradict? Tell us where the benchmark matches your workload, and where it doesn't.
#EventualConsistency #DataEngineering #DataWarehouse #DataInfrastructure #Iceberg
Więcej odcinków z kanału "Eventual Consistency | Your Reality Check on What's Actually Happening in Data"



Nie przegap odcinka z kanału “Eventual Consistency | Your Reality Check on What's Actually Happening in Data”! Subskrybuj bezpłatnie w aplikacji GetPodcast.








