Database School

Hunter Leath, CEO of Archil, explains how they’re building a “universal storage engine” that sits between your apps and S3—making an S3 bucket behave like a fast, POSIX-compatible disk for containers, servers, and even Lambda. Along the way, we dig into how their SSD-backed clusters and custom protocol avoid the usual small-file pain and where this approach shines (and where it doesn’t).Follow Hunter:Twitter/X:  https://twitter.com/jhleathArchil Twitter/X:  https://twitter.com/archildataArchil: https://archil.com/Follow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Intro: Archil Data and “S3 as a disk”01:05 - Hunter’s background and the core pitch02:32 - The real problem: state management (S3 vs block storage)05:02 - SQLite on S3: what the stack looks like07:13 - The missing layer: durable SSD-backed clusters10:14 - Who uses this: unstructured data, CI/CD, Git, agents12:15 - Small files + Git performance and avoiding S3 request explosion16:22 - Why they built a new protocol (NFS vs Luster)20:00 - What gets written to S3: real files in your bucket22:29 - S3 limits, throttling, and the “keep it on SSD” escape hatch25:32 - Multi-cloud + R2, and why regions/latency matter32:10 - Pricing model: “pay only when data is active”34:41 - Tradeoffs: random reads and ultra-low-latency metal37:19 - Storage/compute separation and AI/agent-native workflows43:21 - YC timeline + the marketing challenge of a “universal layer”47:34 - Single-tenant clusters for enterprises and why it’s hard50:27 - Where the company is now, hiring, and how to try it (disk.new)

Hammad Bashir, CTO of Chroma, joins the show to break down how modern vector search systems are actually built from local, embedded databases to massively distributed, object-storage-backed architectures. We dig into Chroma’s shared local-to-cloud API, log-structured storage on object stores, hybrid search, and why retrieval-augmented generation (RAG) isn’t going anywhere.Follow Hammad:Twitter/X:  https://twitter.com/HammadTimeLinkedIn: https://www.linkedin.com/in/hbashirChroma: https://trychroma.comFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 – Introduction From high-school ASICs to CTO of Chroma01:04 – Hammad’s background and why vector search stuck03:01 – Why Chroma has one API for local and distributed systems05:37 – Local experimentation vs production AI workflows08:03 – What “unprincipled data” means in machine learning10:31 – From computer vision to retrieval for LLMs13:00 – Exploratory data analysis and why looking at data still matters16:38 – Promoting data from local to Chroma Cloud19:26 – Why Chroma is built on object storage20:27 – Write-ahead logs, batching, and durability26:56 – Compaction, inverted indexes, and storage layout29:26 – Strong consistency and reading from the log34:12 – How queries are routed and executed37:00 – Hybrid search: vectors, full-text, and metadata41:03 – Chunking, embeddings, and retrieval boundaries43:22 – Agentic search and letting models drive retrieval45:01 – Is RAG dead? A grounded explanation48:24 – Why context windows don’t replace search56:20 – Context rot and why retrieval reduces confusion01:00:19 – Faster models and the future of search stacks01:02:25 – Who Chroma is for and when it’s a great fit01:04:25 – Hiring, team culture, and where to follow Chroma

In this episode of Database School, Aaron Francis sits down with Jordan Tigani, co-founder and CEO of MotherDuck, to break down what DuckDB is, how MotherDuck hosts it in the cloud, and why analytics workloads are shifting toward embedded databases. They dig into Duck Lake, pricing models, scaling strategies, and what it really takes to build a modern cloud data warehouse.Follow Jordan:Twitter/X:  https://twitter.com/jrdntgnLinkedIn: https://www.linkedin.com/in/jordantiganiMotherDuck: https://motherduck.comFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Introduction01:44 - What DuckDB is and why embedded analytics matter04:03 - How MotherDuck hosts DuckDB in the cloud05:18 - Is MotherDuck like the “Turso for DuckDB”?07:38 - Isolated analytics per user and scaling to zero08:51 - The academic origins of DuckDB10:00 - From SingleStore to founding MotherDuck12:28 - Getting fired… and funded 12 days later16:39 - Jordan’s background: Kernel dev, BigQuery, and Product18:36 - Partnering with DuckDB Labs and avoiding a fork20:52 - Why MotherDuck targets startups and the long tail24:22 - Pricing lessons: why $25 was too cheap28:11 - Ducklings, instance sizing, and compute scaling34:16 - How MotherDuck separates compute and storage37:09 - Inside the AWS architecture and differential storage43:12 - Hybrid execution: joining local and cloud data45:14 - Analytics vs warehouses vs operational databases47:41 - Data lakes, Iceberg, and what Duck Lake actually is53:22 - When Duck Lake makes more sense than DuckDB alone56:09 - Who switches to MotherDuck and why58:02 - PG DuckDB and offloading analytics from Postgres1:00:49 - Who should use MotherDuck and why1:03:39 - Hiring plans and where to follow Jordan1:05:01 - Wrap-up

In this episode, Aaron talks with Dennis Magda, author of Just Use Postgres!, about the wide world of modern Postgres, from JSON and full-text search to generative AI, time-series storage, and even message queues. They explore when Postgres should be your go-to tool, when it shouldn’t, and why understanding its breadth helps developers build better systems.Use the code DBSmagda to get 45% off Denis' new book Just Use Postgres!Order Just Use Postgres!Follow Denis:Twitter/X:  https://twitter.com/denismagdaLinkedIn: https://www.linkedin.com/in/dmagdaFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 – Welcome01:28 – Dennis’ Background: Java, JVM, and Databases03:20 – Bridging Application Development & Databases04:05 – Moving Down the Stack: How Dennis Entered Databases07:28 – Apache Ignite, Distributed Systems & the Path to Postgres08:02 – Writing Just Use Postgres!: The Origin Story10:26 – Why a Modern Postgres Book Was Needed11:01 – The Spark That Led to the Book Proposal13:06 – Developers Still Don’t Know What Postgres Can Do15:40 – Connecting With Manning & Refining the Book Vision16:38 – What Just Use Postgres! Covers17:40 – The Book’s Core Thesis: The Breadth of Postgres19:50 – Favorite Use Cases & Learning While Writing20:30 – When to Use Postgres for Non-Relational Workloads23:08 – Full Text Search in Postgres Explained29:31 – When Not to Use Postgres (Pragmatism Over Fanaticism)34:01 – Using Postgres as a Message Queue42:09 – When Message Queues Outgrow Postgres48:10 – Postgres for Generative AI (PGVector)55:34 – Dennis’ 14-Month Writing Process01:00:50 – Who the Book Is For01:04:10 – Where to Follow Dennis & Closing Thoughts

In this episode, Gajus Kuizinas, co-founder and CTO of Contra, joins Aaron to talk about building the engineering world you want to live in, from strict runtime-validated SQL with Slonik to creating high-ownership engineering cultures. They dive into developer experience, runtime assertions, SafeQL, and even “Loom-driven development,” a powerful review process that lets teams move fast without breaking things.Follow Gajus:Twitter/X:  https://twitter.com/kuizinasSlonk: https://github.com/gajus/slonikScaling article: https://gajus.medium.com/lessons-learned-scaling-postgresql-database-to-1-2bn-records-month-edc5449b3067Follow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 – Introduction01:03 – Meet Gajus and Contra01:48 – What Contra does and how it’s different05:34 – Why Slonik exists & early career origins07:47 – The early Node.js era and frustrations with ORMs09:50 – SQL vs abstractions and the case for raw SQL10:35 – Template tags and the breakthrough idea12:03 – Strictness, catching errors early & data shape guarantees13:37 – Runtime type checking, Zod, and performance debates16:02 – SafeQL and real-time schema linting17:01 – Synthesizing Slonik’s philosophy21:29 – Handling drift, static types vs reality22:52 – Defining schemas per-query & why it matters27:59 – Integrating runtime types with large test suites31:00 – Scaling the team and performance tradeoffs33:41 – Runtime validation cost vs developer productivity35:21 – Real drift examples from payments & external APIs38:21 – User roles, data shape differences & edge cases39:51 – Integration test safety & catching issues pre-deploy40:52 – Contra’s engineering culture41:47 – Why traditional PR reviews don’t scale43:22 – Introducing Loom-Driven Development45:12 – How looms transformed the review process52:38 – Using GetDX to measure engineering friction53:07 – How the team uses AI (Claude, etc.)56:26 – Closing thoughts on DX and engineering philosophy58:05 – Contra needs Postgres experts59:00 – Where to find Gajus

In this episode, Aaron Francis talks with Simon Eskildsen, co-founder and CEO of TurboPuffer, about building a high-performance search engine and database that runs entirely on object storage. They dive deep on Simon's time as an engineer at Shopify, database design trade-offs, and how TurboPuffer powers modern AI workloads like Cursor and Notion.Follow Simon:Twitter: https://twitter.com/SirupsenLinkedIn: https://ca.linkedin.com/in/sirupsenTurbopuffer: https://turbopuffer.comFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters00:00 - Introduction01:11 - Simon’s background and time at Shopify03:01 - The Rails glory days and early developer experiences04:55 - From PHP to Rails and joining Shopify06:14 - The viral blog post that led to Shopify09:03 - Discovering engineering talent through GitHub10:06 - Scaling Shopify’s infrastructure to millions of requests per second12:47 - Lessons from hypergrowth and burnout14:46 - Life after Shopify and “angel engineering”16:31 - The Readwise problem and discovering vector embeddings18:22 - The high cost of vector databases and napkin math19:14 - Building TurboPuffer on object storage21:20 - Landing Cursor as the first big customer23:00 - What TurboPuffer actually is25:26 - Why object storage now works for databases28:37 - How TurboPuffer stores and retrieves data31:06 - What’s inside those S3 files33:02 - Explaining vectors and embeddings35:55 - How TurboPuffer v1 handled search38:00 - Transitioning from search engine to database44:09 - How Turbopuffer v2 and v3 improved performance47:00 - Smart caching and architecture optimizations49:04 - Trade-offs: high write latency and cold queries51:03 - Cache warming and primitives52:25 - Comparing object storage providers (AWS, GCP, Azure)55:02 - Building a multi-cloud S3-compatible client57:11 - Who TurboPuffer serves and the scale it runs at59:31 - Connecting data to AI and the global vision1:00:15 - Company size, scale, and hiring1:01:36 - Roadmap and what’s next for TurboPuffer1:03:10 - Why you should (or shouldn’t) use TurboPuffer1:05:15 - Closing thoughts and where to find Simon

Aaron talks with Ovais Tariq, co-founder and CEO of Tigris Data and former Uber engineer who helped scale one of the world’s largest distributed systems. They discuss Uber’s hyperscale infrastructure, what it takes to build an S3-compatible object store from scratch, and how distributed storage is evolving for the AI era.Follow Ovais:Twitter: https://twitter.com/ovaistariqLinkedIn: https://www.linkedin.com/in/ovaistariqTigris: https://www.tigrisdata.comFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Introduction and overview of the episode01:35 - Ovais’s background and introduction to Tigris03:00 - Building distributed databases and infrastructure at Uber06:00 - Uber’s in-house philosophy and massive data scale09:00 - Hardware, power density, and talking to chip manufacturers12:00 - Learning curve of scaling hardware and data centers14:00 - The Halloween outage and lessons from Cassandra16:00 - Building data centers across the world for Uber17:00 - Founding Tigris and the vision for global storage18:45 - How Tigris differs from AWS S320:00 - The architecture of Tigris: caching, metadata, and replication32:00 - Why Tigris uses FoundationDB and its reliability36:00 - Managing global and regional metadata38:00 - How Tigris dynamically moves and caches data41:30 - Building their own data centers and backbone43:45 - Specialized storage for AI workloads46:00 - Small file optimization and real-world use cases49:00 - Snapshots, forking, and agentic AI workflows51:00 - How AI transformed Tigris’s customer base54:00 - Partnership with Fly.io and the distributed cloud ecosystem57:00 - Growth, customers, and focus on media and AI companies59:00 - What’s next for Tigris: distributed file system plans1:01:00 - Technical challenges and building trust in durability1:03:00 - Call to action: try Tigris and upcoming snapshot feature1:05:00 - Advice for engineers leaving big companies to start something new1:06:30 - Where to find Ovais online and closing remarks

In this episode of Database School, Aaron talks with Preston Thorpe, a senior engineer at Turso who is currently incarcerated, about his incredible journey from prison to rewriting SQLite in Rust. They dive deep into concurrent writes, MVCC, and the challenges of building a new database from scratch while discussing redemption, resilience, and raw technical brilliance.Follow Preston and Turso:LinkedIn: https://www.linkedin.com/in/PThorpe92Preston's Blog: https://pthorpe92.devGitHub: https://github.com/PThorpe92Turso: https://turso.techFollow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Intro and Preston’s story02:13 - How Preston learned programming in prison06:06 - Making his parents proud and turning life around09:01 - Getting his first job at Unlock Labs10:47 - Discovering Turso and contributing to open source12:53 - From contributor to senior engineer at Turso22:27 - What Preston works on inside Turso24:00 - Challenges of rewriting SQLite in Rust26:00 - Why concurrent writes matter27:57 - How Turso implements concurrent writes35:02 - Maintaining SQLite compatibility37:03 - MVCC explained simply43:40 - How Turso handles MVCC and logging46:03 - Open source contributions and performance work46:23 - Implementing live materialized views50:55 - The DBSP paper and incremental computation52:55 - Sync and offline capabilities in Turso56:45 - Change data capture and future possibilities1:02:01 - Implementing foreign keys and fuzz testing1:06:02 - Rebuilding SQLite’s virtual machine1:08:10 - The quirks of SQLite’s codebase1:10:47 - Preston’s upcoming release and what’s next1:14:02 - Gratitude, reflection, and closing thoughts

In this episode, Aaron talks with Joran Greef, CEO and creator of TigerBeetle, the world’s first financial transactions database. Joran takes us on a deep dive of on how TigerBeetle brings double-entry accounting principles directly into the database layer to achieve extreme correctness, performance, and fault tolerance at scale.Follow Joran and TigerBeetle:Twitter/X: https://twitter.com/jorandirkgreefWebsite: https://tigerbeetle.comGitHub: https://github.com/tigerbeetle/tigerbeetleTiger Style: https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TIGER_STYLE.mdYouTube: https://www.youtube.com/@UC3TlyQ3h6lC_jSWust2leGg Follow Aaron:Twitter/X:  https://twitter.com/aarondfrancis Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g  (Subscribe today)LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Introduction and crossover between accounting and databases01:50 - Meet Joran Greef and the origins of TigerBeetle02:55 - What makes TigerBeetle different from general purpose databases04:38 - The founding story and the 5,000-year history of transactions07:42 - How modern commerce became highly transactional10:06 - Recognizing the limits of general purpose databases13:18 - From distributed systems to payment infrastructure17:01 - Discovering bottlenecks in traditional database performance19:58 - Why traditional databases can’t scale for microtransactions23:05 - Introducing double-entry accounting concepts25:20 - How double-entry accounting mirrors database design31:35 - Modeling ledgers and event sourcing in Tiger Beetle35:02 - Why TigerBeetle outperforms Postgres and MySQL40:05 - Batching transactions for massive throughput47:09 - Client-side batching and zero-copy efficiency50:04 - Handling contention and concurrency internally56:03 - Ensuring correctness and atomicity in transactions57:17 - Designing for mission-critical systems and reliability1:00:50 - Building safety through deterministic simulation testing1:04:55 - Detecting and recovering from storage faults1:10:00 - How TigerBeetle prevents data corruption1:17:01 - Distributed replication and self-healing data1:20:08 - Who’s using TigerBeetle and how it’s structured as a company1:24:01 - How to learn more and get involved with TigerBeetle1:26:15 - Closing thoughts and where to find Joran online

Sam Lambert, my former boss at PlanetScale, talks to me about PlanetScale moving from a MySQL company to now also having a Postgres offering. Sam shares why PlanetScale decided to move to Postgres, how MySQL and Postgres are different at a technical level, and how the change has impacted the company culture. Stay to the end for a special surprise!PlanetScale Metal Episode: https://youtu.be/3r9PsVwGkg4Join the waitlist to be notified of the MySQL for Developers release on Database School: https://databaseschool.com/mysqlFollow Sam: PlanetScale: https://planetscale.comTwitter: https://twitter.com/isamlambertFollow Aaron:Twitter:  https://twitter.com/aarondfrancis LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Database School: https://databaseschool.comDatabase School YouTube Channel: https://www.youtube.com/@UCT3XN4RtcFhmrWl8tf_o49g (Subscribe today)Chapters:00:00 - Inaugural episode on this channel01:46 - Introducing Sam Lambert and his background03:04 - How PlanetScale built on MySQL and Vitess06:10 - Explaining the layers of PlanetScale’s architecture09:57 - Node lifecycles, failover, and operational discipline12:02 - How Vitess makes sharding work14:21 - PlanetScale’s edge network and resharding19:02 - Why downtime is unacceptable at scale20:04 - From Metal to Postgres: the decision process23:06 - Why Postgres vibes matter for startups27:04 - How PlanetScale adapted its stack for Postgres34:38 - Entering the Postgres ecosystem and extensions41:02 - Permissions, security, and reliability trade-offs45:04 - Building Ni: a Vitess-style system for Postgres53:33 - Why PlanetScale insists on control for reliability1:02:05 - Competing in the broader Postgres landscape1:08:33 - Why PlanetScale stays “just a database”1:12:33 - What GA means for Postgres at PlanetScale1:17:43 - Call to action for new Postgres users1:18:49 - Surprise!1:22:21 - Wrap-up and where to find Sam