Grumpy old Coders: Ep19 - Scaling pains

David and Thomas dive into the inner workings of the Transformer architecture behind modern Large Language Models. They break down foundational concepts including vectors, matrices, function approximations, artificial neural networks, and backpropagation. Supported by a live Python code demonstration, they walk through tokenization, word embeddings, sinusoidal positional encodings, and Query-Key-Value attention dynamics. The episode wraps up with a discussion on scaling context windows, KV caching, and the philosophical debate around AI consciousness versus human self-awareness. Outline: 00:21 – Episode intro, podcast banter, & topic overview 02:37 – Mathematical foundations: Vectors, matrices, & functions 06:18 – Artificial Neural Networks: Layers, weights, & backpropagation 11:42 – ANN practical example: Image classification mechanics 14:51 – Tokenization & vocabulary constraints 18:40 – Word embeddings & GloVe vector spaces 21:00 – Positional encoding & sinusoidal matrices 27:00 – Self-attention components: Queries (Q), Keys (K), and Values (V) 31:37 – Live Python code walkthrough & script execution 37:25 – Attention matrix calculations & Softmax normalization 45:56 – Multi-head attention & multi-stage learning dynamics 49:44 – Large Language Model context window scaling 55:13 – KV caching & inference optimization 58:28 – AI consciousness debate 1:14:24 – Guardrails, sampling temperature, & LLM watermarking 1:20:51 – Closing thoughts and sign-off

David and Thomas recap their multi-country motorcycle journey through Germany, Austria, Slovenia, and Italy. They share key moments from the road, including navigating winding mountain serpentines. Along the way, they reflect on awkward luggage setups and unpredictable alpine weather. The episode wraps up with a discussion on the tech job market, the realities of AI "vibe coding" versus software engineering fundamentals, and preliminary planning for their next road trip. Outline: 00:20 – Episode intro, camera/mic troubleshooting, & tour premise 01:00 – Bike comparison: Suzuki SV650 vs. BMW R 1250 GS 03:11 – Lake Chiemsee stop, rain, and duck watching 06:08 – Crossing into Austria via Kufstein & taking biker photos 08:20 – Triassic Park landmark & tour snacking habits 10:32 – Riding through Slovenia: Serpentines 15:00 – Wörthersee stopover 20:04 – Tom's leg injury 24:25 – High-altitude dining & Slovenian road marking confusion 29:13 – Luggage setup challenges 35:00 – Nolan helmet visor breakdown 40:14 – Trattoria al Norgé: Homemade pasta & Italian hospitality 45:37 – Narrow mountain pass & wet bitumen hazards 52:30 – Staying in Innervillgraten & Innsbruck hospital visit 1:01:54 – Returning to Germany 1:11:00 – Tech industry state, LLMs, & "vibe coding" vs. engineering 1:19:10 – Future trip planning 1:22:50 – Closing thoughts and sign-off

David and Thomas dive into the critical role of context management and agentic memory with large language models. They explore the core mechanics of AI agents, breaking down the differences between low-level tools, Model Context Protocol (MCP) servers, and high-level skills, before tackling the token cost challenges of stateless LLMs. The hosts analyze human-inspired cognitive strategies like context "compaction" and background "dreaming". The episode closes with thoughts on codebase visualization using Graphify and Obsidian, and a humorous nod to Inception-style agent recursion. Outline: 00:21 – Heatwave talk & audio troubleshooting 07:16 – Chatbots vs. AI agents 12:20 – Defining tools, MCP, and skills 20:51 – Context boundaries & compaction 33:12 – Cognitive concepts and "dreaming" 40:14 – Exploring several tools 1:11:15 – Closing

After a year-long break, David and Thomas return to discuss CHAOSS (Community Health Analytics in Open Source Software), a Linux Foundation framework for evaluating open source project health. They explore metrics like compliance/security, governance, and community activity. The hosts emphasize that while metrics provide valuable data, community trust and authentic collaboration matter most. They also share motorcycle adventures—David's new Suzuki SV650 and their Black Forest tour. The episode closes with plans for more frequent releases and inviting guest speakers. Outline: 00:00 – Reunion & motorcycle talk 08:20 – What is CHAOSS? 15:26 – Metric groups (compliance, governance, community) 22:27 – Bus factor & sustainability 25:00 – GrimoireLab monitoring tool 28:14 – When to adopt CHAOSS 34:26 – Future plans & closing

In this episode, David and Thomas delve into the concept of Product-Led Growth (PLG). They discuss the differences between sales-led and product-led approaches, emphasizing the importance of user experience and collaboration across teams. The conversation highlights the benefits of PLG, particularly for startups, and the necessity of understanding customer needs and feedback to ensure product success. In this conversation, Thomas and David explore the intricacies of product-led growth (PLG) and the journey users take from being strangers to becoming champions of a product. Chapters: 00:00 Introduction to Product-Led Growth 02:55 Understanding Go-to-Market Strategies 10:05 The Role of User Experience in PLG 15:09 Benefits and Challenges of PLG 21:16 Collaboration Across Teams for Success 36:36 The Genesis of Open Source Products 37:58 Understanding Customer Acquisition Costs 38:55 The Product-Led Growth Framework 40:37 Transitioning from Strangers to Explorers 43:41 The Evaluation Stage: From Stranger to Explorer 46:51 Activation: The Moment of Value Realization 51:20 From Explorer to Beginner: The Activation Journey 53:11 The Importance of Documentation and Training 56:47 Onboarding Experiences and User Guidance 01:01:29 Driving Adoption and Cultivating Regular Users 01:06:41 The Role of Champions in Product Advocacy 01:08:00 The Role of Champions in Product Growth 01:10:11 Understanding the Flywheel Effect in SaaS 01:14:02 Brand Loyalty and User Experience 01:15:56 The Competitive Landscape of Cloud Services 01:19:02 Open Source vs. Proprietary Software Keywords: Product-Led Growth, Go-to-Market Strategy, User Experience, Sales, Marketing, SaaS, Customer Success, PLG Benefits, Collaboration, Startup, open source, product-led growth, customer acquisition, evaluation stage, activation, onboarding, user experience, documentation, champions, SaaS, Product Growth, SaaS, Champions, Flywheel Effect, Brand Loyalty, Cloud Services, Open Source, PLG, Software Licensing

In this episode, Thomas and David discuss the evolving landscape of e-mobility, focusing on electric vehicles (EVs) and their implications for consumers and the environment. They explore various types of electric vehicles, including hybrids and fully electric models, and delve into battery technology, efficiency, and the cost of ownership. The conversation also touches on the charging infrastructure, everyday usage experiences, and the longevity of EV batteries. Additionally, they examine the competitive landscape, particularly the rise of Chinese manufacturers in the EV market. Chapters: 00:00 Types of Electric Vehicles 06:09 Efficiency of Electric Vehicles 12:28 Comparing Electric and Traditional Vehicles 19:30 Pricing and Competition in the Market 25:17 Range and Charging Infrastructure 29:12 Benefits and Considerations of E-Mobility 34:32 Understanding Charging Options 36:26 Availability and Pricing of Charging Stations 44:02 Different Prices for Charging 46:38 Charging at Home and Battery Lifespan 53:18 Lifespan of Electric Car Batteries 57:30 Replacing the Battery 01:01:07 Efficiency and Maintenance of Electric Cars 01:03:23 Financial Considerations and Environmental Impact 01:13:28 Chinese Car Manufacturers in the Global Market Keywords: e-mobility, electric vehicles, battery technology, charging infrastructure, environmental impact, cost considerations, market trends, efficiency, hybrid vehicles, Chinese competition

In this episode, David and Thomas discuss their use of a large language model to build a chatbot based on documentation. They explain the concept of prompt engineering and how it can be used to guide the model's responses. They also introduce the concept of retrieval augmented generation (RAG). They also discuss the use of knowledge base articles and documentation to influence the behavior of LLMs. Additionally, they touch on the use of graph databases and embeddings to identify redundancies in documentation and improve search results. The conversation concludes with a discussion on the challenges and limitations of working with LLMs.

Merry Christmas (or just a happy holiday season)! Thomas and David discuss how technology is part of a wider definition of culture and how techies can contribute to technical education of kids.

Thomas and David talk about Technical Enablement, how David came to work on Technical Enablement projects, how it can help compensating the pains of scaling quickly, Learning Experience Design and more.

Thomas and David talk about how to produce a podcast. You will learn which equipment is needed, which software can be used and how the prep work and post-production looks like. Have fun!