Building a Serverless Streaming Platform

You can't manage what you don't measure, and this includes your cloud costs. But how detailed should this measurement be? And how will the data translate into impact? I sat down with Adam Shugar, co-founder and CTO of Dashdive, to discuss his approach to cloud costs. He shared his advice, not only on cost cutting but also technology, growing a startup, the importance of community and more.

Bill Tarr has the most interesting job in the world. He and his team of SaaS Evangelists at the AWS SaaS Factory work with companies large and small to help them build SaaS on AWS. In this conversation, we discuss the different ways technology and business interact when building SaaS - from cool technical options available only to SaaS companies to the importance of knowing the cost of a tenant. This hour is packed with advice for both the technical and the business side of building a viable SaaS. Links mentioned in the show: Prime Day team on observability: https://www.youtube.com/watch?v=fOYOvp6X10g Bill Tarr on SaaS Architecture Pitfalls: https://www.youtube.com/watch?v=fOYOvp6X10g WrapStream's architecture: https://youtu.be/ceWOw8rxDOM?si=uECCtW0SJ1Sffqn_ Alcion's zero cost multi-region architecture: https://youtu.be/kJECSpVwM7Q?si=_j9LZWdoA5xpFC3g

Ryan Worl, co-founder and CTO of WarpStream, is on a mission to re-engineer fundamental infrastructure on top of S3. Starting with metrics at Uber, continuing with Husky - DataDog's platform for events, logs and everything except metrics and then... Ryan thought "it will be cool to re-build Kafka on S3", reached out to other developers to hear their thoughts... and ended up building WarpStream. In this episode we talk about Ryan's journey, how S3 can fundamentally change the way we design our infrastructure, and why it is a good idea to go in that direction. We talk about S3 vs S3 express. What will be the perfect serverless experience for streams. And finally, tune in at minute 50:00 to get excellent advice on how a SaaS provider should use logs.

We invited Anna Povzner, director of engineering at Confluent, to the show to discuss Kora. Kora is a Cloud Native platform based on Apache Kafka, which her team at Confluent built. Anna and her team recently published a paper about Kora in VLDB 2023, and it won the industry's best paper award. Kora VLDB paper: https://www.vldb.org/pvldb/vol16/p3822-povzner.pdf In our conversation, we discussed: - Capacity abstractions for serverless Kafka - Reconciling design constraints into a cohesive architecture - Key components of the architecture: Rebalances, quotas, cells. - How to drive deep technical innovation at a fast-growing company - Delivering low and predictable latency with Apache Kafka

In the last year or two I started hearing a lot about cell-based architectures. Usually in the form of “We had a lot of issues scaling our infrastructure, but then we moved to cell-based architectures” and “I wish I’ve learned about cell based architectures earlier, it would have saved me a lot of pain”. As a result, I’ve wanted to share knowledge about cell-based architectures with this community for a while now. I was lucky that Eno Thereska called me and suggested to do just that! Eno, currently at Alcion, is one of the most impressive technical leaders I’ve head the pleasure of working with. He has deep theoretical knowledge that he knows how to use for very practical technical solutions. And in this presentation and discussion, he shares both theory and practical advice. We discussed everything from the basics of cell based architectures, their benefits all the way to different heuristics for assigning tenants to cells. Papers and talks we discussed: AWS Fargate under the hood: https://www.youtube.com/watch?v=Hr-zOaBGyEA Doordash - Journey to cell-based micro services architecture: https://www.youtube.com/watch?v=ReRrhU-yRjg Slack’s Migration to Cellular Architecture: https://slack.engineering/slacks-migration-to-a-cellular-architecture/ Kora: A cloud-native event streaming platform for Kafka: https://www.vldb.org/pvldb/vol16/p3822-povzner.pdf

Krishna Raman has many years of experience building platforms for developers. Now he's applying this experience at Delta Streams to build a serverless platform for stream processing in SQL. In our conversation, we discussed the serverless developer experience, some of the secret sauce behind Delta Stream's Flink Operator, how to deliver a bring-your-own-account SaaS, K8s network policies and why they matter and when you shouldn't use K8s at all.

You are a founding engineer at a SaaS startup. You built the MVP, and to everyone's great delight - usage is picking up. What's next? In this episode, Jeffrey shares how a messy MVP can gradually evolve into a scalable SaaS product. We discuss the critical design decisions engineers have to make in the early days of building the product: Tabs or spaces, pooled tenants or siloed, Micro-services or monolith... And who makes these decisions. Jeffrey Sherman is a staff engineer, scalability, and performance lead at Active Campaign - and was recently recognized as SaaS Developer Hero for his contributions to this community. Jeffery has his own podcast, @NeverRewrite , where you can find him with more good advice and funny stories every single week. P.S. I had a bit of construction going on while recording. I tried my best, but you'll hear hammers in the background. So sorry!

Did you know that Postgres that lets you write data that you can’t query? Events that will show up in the write-ahead log (WAL) of the database but not in any table. Gunnar Morling, senior staff engineer at Decodable and world expert on change data capture, walks us through data capture basics, this not-new but little-known feature, and dives into three key use cases (my favorite is the audit log) plus a bonus example that goes deeper into Postgres internals. We discuss these blogs and examples in the video: - https://www.infoq.com/articles/wonder... - https://www.morling.dev/blog/insatiab... - https://github.com/decodableco/exampl...

Andrew Atkinson took a Rails web application that was struggling with load, and optimized it to handle over 9000 HTTP requests per second with an average latency of 35ms end to end. Handling a much higher load, on a smaller RDS instance, with lower latencies. He then shared his expertise by writing a book: "High-Performance Postgres with Rails." Andrew and I discussed Postgres performance, scalability, design patterns, valuable tools, his career, and some Rails. I tackled Andrew with many of the biggest challenges SaaS developers encounter with Postgres, and Andrew had an answer to everything. For me, the highlights were his explanation of the N+1 problem, sharding patterns, the use of pgcopydb project to move a noisy neighbor to another DB, and the secret for scaling on a single instance. Useful links: - The book: https://pgrailsbook.com - SaaS Developer Slack - where Andrew answers Postgres questions and will share a discount for his book: http://launchpass.com/all-about-saas Tools we mentioned: - Easier multi-tenancy for Rails: https://github.com/ErwinM/acts_as_tenant - For read replicas prior to Rails 6.0: https://github.com/OutOfOrder/multidb - The fastest way to copy a DB or part of: https://github.com/dimitri/pgcopydb

Ensuring your production system is well-behaved is table stakes for any SaaS. The space around monitoring and alerting is complex and moving fast, so it is too easy to end up with results that are worse than useless. Alert fatigue is real, and developers must learn to avoid this. Shahar and Tal, founders of Keep and active SaaS Developer community members, joined me to discuss - why observability is so complicated, how alerts fit in, and many best practices for alerting. If you feel like you have a good handle on your alerts today, you should skip to minute 23:40, where we talk about the future. CI/CD for alerts, semantic layer for alerts, and how AI will recommend alerts + ways to resolve them.