Episode 477: Josef Strzibny on Self Hosting Applications

In this episode of Software Engineering Radio, Srujana Merugu, an AI researcher with decades of experience, speaks with host Priyanka Raghavan about building LLM-based applications. The discussion begins by clarifying essential concepts like generative vs. predictive AI, pre-training vs. fine-tuning, and the transformer architecture that powers modern LLMs. Srujana explains diffusion models and vision transformers, highlighting how multimodal AI is reshaping content creation. The conversation then moves to practical aspects—where LLMs make sense, where they don't, and a decision framework for evaluating use cases. They explore common application patterns such as retrieval-augmented generation (RAG) and agentic architectures, breaking down components like planners, orchestrators, memory, and tools. Key considerations for model selection, evaluation metrics, and safety guardrails are discussed in depth. The episode also touches on prompting strategies, automated prompt optimization, and emerging trends like multi-sensory AI and the "Internet of Senses." Finally, Srujana shares tips on staying current in a fast-moving AI landscape and emphasizes lifelong learning and curated knowledge sources.

Philip Kiely, software developer relations lead at Baseten, speaks with host Jeff Doolittle about multi-agent AI, emphasizing how to build AI-native software beyond simple ChatGPT wrappers. Kiely advocates for composing multiple models and agents that take action to achieve complex user goals, rather than just producing information. He explains the transition from off-the-shelf models to custom solutions, driven by needs for domain-specific quality, latency improvements, and economic sustainability, which introduces the engineering challenge of inference engineering. Kiely stresses that AI engineering is primarily software engineering with new challenges, requiring robust observability and careful consideration of trust and safety through evals and alignment. He recommends an approach of iterative experimentation to get started with multi-agent AI systems. Brought to you by IEEE Computer Society and IEEE Software magazine.

Flavia Saldanha, a consulting data engineer, joins host Kanchan Shringi to discuss the evolution of data engineering from ETL (extract, transform, load) and data lakes to modern lakehouse architectures enriched with vector databases and embeddings. Flavia explains the industry's shift from treating data as a service to treating it as a product, emphasizing ownership, trust, and business context as critical for AI-readiness. She describes how unified pipelines now serve both business intelligence and AI use cases, combining structured and unstructured data while ensuring semantic enrichment and a single source of truth. She outlines key components of a modern data stack, including data marketplaces, observability tools, data quality checks, orchestration, and embedded governance with lineage tracking. This episode highlights strategies for abstracting tooling, future-proofing architectures, enforcing data privacy, and controlling AI-serving layers to prevent hallucinations. Saldanha concludes that data engineers must move beyond pure ETL thinking, embrace product and NLP skills, and work closely with MLOps, using AI as a co-pilot rather than a replacement. Brought to you by IEEE Computer Society and IEEE Software magazine.

Dave Thomas, author of The Pragmatic Programmer, The Manifesto for Agile Software Development, Programming Ruby, Agile Web Development with Rails, Programming Elixir, Simplicity, and co-founder of the Pragmatic Bookshelf, speaks with SE Radio host Gavin Henry about building infrastructure for eBooks. They discuss what an eBook is, the various formats, what infrastructure is needed to build them, how an author writes an book, the history of the Pragmatic Bookshelf, how they have evolved, how to handle links within eBooks, why humans are so important in the writing process, and why AI can help with your writing -- once you've written your content. Thomas discusses PDFs, eBooks, Mobi files, ePub files, CI/CD pipelines, WYSWYG, Markdown files, Pragmatic Markup Language, embedding code, AI agents, images, printing PDFs, JVMs, Java, jRuby, and how Markdown won the plain text writing format wars. Brought to you by IEEE Computer Society and IEEE Software magazine.

Jennings Anderson, a Software Engineer with Meta Platforms, and Amy Rose, the Chief Technology Officer at Overture Maps Foundation, speak with host Gregory M. Kapfhammer about the Overture Maps project, which creates reliable, easy-to-use, and interoperable open map data. After exploring the foundations of geospatial information systems, Gregory and his guests dive deep into the implementation of Overture Maps through features like the Global Entity Reference System (GERS). In addition to discussing the organizational structure of the Overture Maps Foundation and the need for a unified database of geospatial data, Jennings and Amy explain how to implement applications using data from Overture Maps. Brought to you by IEEE Computer Society and IEEE Software magazine.

Mark Williamson, CTO of Undo, joins host Priyanka Raghavan to discuss AI-assisted debugging. The conversation is structured around three main objectives: understanding how AI can serve as a debugging assistant;  examining AI-powered debugging tools; exploring whether AI debuggers can independently find and fix bugs. Mark highlights how AI can support debugging with its ability to analyze vast amounts of data, narrow down issues, and even generate tests. From there, the discussion turns to AI debugging tools, with a particular look at ChatDBG's strengths and limitations, with a peek at time travel debugging. In the final segment, they consider several real-world scenarios and evaluate the feasibility and practicality of AI acting autonomously in debugging. Brought to you by IEEE Computer Society and IEEE Software magazine.

Sourabh Satish, CTO and co-founder of Pangea, speaks with SE Radio's Brijesh Ammanath about prompt injection. Sourabh begins with the basic concepts underlying prompt injection and the key risks it introduces. From there, they take a deep dive into the OWASP Top 10 security concerns for LLMs, and Sourabh explains why prompt injection is the top risk in this list. He describes the $10K Prompt Injection challenge that Pangea ran, and explains the key learnings from the challenge. The episode finishes with discussion of specific prompt-injection techniques and the security guardrails used to counter the risk. Brought to you by IEEE Computer Society and IEEE Software magazine.

Kacper Łukawski, a Senior Developer Advocate at Qdrant, speaks with host Gregory M. Kapfhammer about the Qdrant vector database and similarity search engine. After introducing vector databases and the foundational concepts undergirding similarity search, they dive deep into the Rust-based implementation of Qdrant. Along with comparing and contrasting different vector databases, they also explore the best practices for the performance evaluation of systems like Qdrant. Kacper and Gregory also discuss topics such as the steps for using Python to build an AI-powered application that uses Qdrant. Brought to you by IEEE Computer Society and IEEE Software magazine.

Florian Gilcher, co-founder of Ferrous Systems and the Rust Foundation, speaks with host Giovanni Asproni about the application of Rust in mission- and safety-critical systems. The discussion starts with a brief overview of such systems, and an introduction to Rust, emphasizing aspects that make it well-suited for critical environments. Florian and Giovanni then discuss how Rust compares to C and C++ — two widely used languages in this sector. They proceed to outline important factors that companies should consider when assessing whether to move from C or other languages to Rust. The episode also touches on Ferrocene, an open-source Rust toolchain qualified for safety- and mission-critical systems, which was developed and supported by Ferrous Systems. The conversation ends with some reflections on the future of Rust for mission- and safety-critical applications. Brought to you by IEEE Computer Society and IEEE Software magazine.

Amey Desai, the Chief Technology Officer at Nexla, speaks with host Sriram Panyam about the Model Context Protocol (MCP) and its role in enabling agentic AI systems. The conversation begins with the fundamental challenge that led to MCP's creation: the proliferation of "spaghetti code" and custom integrations as developers tried to connect LLMs to various data sources and APIs. Before MCP, engineers were writing extensive scaffolding code using frameworks such as LangChain and Haystack, spending more time on integration challenges than solving actual business problems. Desai illustrates this with concrete examples, such as building GitHub analytics to track engineering team performance. Previously, this required custom code for multiple API calls, error handling, and orchestration. With MCP, these operations can be defined as simple tool calls, allowing the LLM to handle sequencing and error management in a structured, reasonable manner. The episode explores emerging patterns in MCP development, including auction bidding patterns for multi-agent coordination and orchestration strategies. Desai shares detailed examples from Nexla's work, including a PDF processing system that intelligently routes documents to appropriate tools based on content type, and a data labeling system that coordinates multiple specialized agents. The conversation also touches on Google's competing A2A (Agent-to-Agent) protocol, which Desai positions as solving horizontal agent coordination versus MCP's vertical tool integration approach. He expresses skepticism about A2A's reliability in production environments, comparing it to peer-to-peer systems where failure rates compound across distributed components. Desai concludes with practical advice for enterprises and engineers, emphasizing the importance of embracing AI experimentation while focusing on governance and security rather than getting paralyzed by concerns about hallucination. He recommends starting with simple, high-value use cases like automated deployment pipelines and gradually building expertise with MCP-based solutions. Brought to you by IEEE Computer Society and IEEE Software magazine.