0:00
1:22:27
David and Thomas dive into the inner workings of the Transformer architecture behind modern Large Language Models. They break down foundational concepts including vectors, matrices, function approximations, artificial neural networks, and backpropagation. Supported by a live Python code demonstration, they walk through tokenization, word embeddings, sinusoidal positional encodings, and Query-Key-Value attention dynamics. The episode wraps up with a discussion on scaling context windows, KV caching, and the philosophical debate around AI consciousness versus human self-awareness.
Outline:
00:21 – Episode intro, podcast banter, & topic overview
02:37 – Mathematical foundations: Vectors, matrices, & functions
06:18 – Artificial Neural Networks: Layers, weights, & backpropagation
11:42 – ANN practical example: Image classification mechanics
14:51 – Tokenization & vocabulary constraints
18:40 – Word embeddings & GloVe vector spaces
21:00 – Positional encoding & sinusoidal matrices
27:00 – Self-attention components: Queries (Q), Keys (K), and Values (V)
31:37 – Live Python code walkthrough & script execution
37:25 – Attention matrix calculations & Softmax normalization
45:56 – Multi-head attention & multi-stage learning dynamics
49:44 – Large Language Model context window scaling
55:13 – KV caching & inference optimization
58:28 – AI consciousness debate
1:14:24 – Guardrails, sampling temperature, & LLM watermarking
1:20:51 – Closing thoughts and sign-off
Weitere Episoden von „Grumpy old Coders“
Verpasse keine Episode von “Grumpy old Coders” und abonniere ihn in der kostenlosen GetPodcast App.
