The Monte Carlo Method

We talk about the history of artificial neural networks and attempt to explain some of their workings with the help of our friend Bob Farzin.  Neurons: How the Brain Communicates | Mental Health AmericaA LOGICAL CALCULUS OF THE IDEAS IMMANENT IN NERVOUS ACTIVITY*THE PERCEPTRON: A PROBABILISTIC MODEL FOR INFORMATION STORAGE AND ORGANIZATION IN THE BRAIN1 (Rosenblatt)The Rosenblatt Perceptron AlgorithmMinsky-and-Papert-Perceptrons.pdfLearning representations by back-propagating errorsBackpropagation Applied to Handwritten Zip Code Recognition - Y. LeCunImageNet Classification with Deep Convolutional Neural Networks (aka Alexnet)[1706.03762] Attention Is All You Need (Transformers)Send us Fan Mail

In this episode we explore the likelihood of existential risk from AI.  We discuss the viewpoints of doomers and skeptics, and some of the scenarios that could lead to catastrophic outcomes.  We admire the takes of the third most famous Michael Jordan. We conclude that not ALL of us will die, but maybe some of us.Michael I. Jordan on AI: “Superintelligence is science fiction”AI as Normal TechnologyDario Amodei — We Must Pace the Frontier'A billion deaths': Bill Gates warns about AI in the wrong hands, saying a kill switch and self-regulation won’t be enough | FortuneIs the AI existential threat real—and can regulatory action prevent it? | BrookingsGenerative design of bacteriophages with genome language models | ScienceHow Might Artificial Intelligence Affect the Risk of Nuclear War?Michael I. Jordan on AI: “Superintelligence is science fiction”AI ‘godfather’ Yann LeCun has ‘zero concerns’ about human extinction, says Anthropic CEO Dario Amodei is ‘deluded’ | FortuneAI 2027 - Rogue Replication Timeline — LessWrongSend us Fan Mail

This week we conclude our discussion of dynamic memory allocation with a discussion of the history and implementation of garbage collectors.  There are no mentions of raccoons.Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part IUniprocessor Garbage Collection Techniques?cpython/InternalDocs/garbage_collector.md at mainGeneration Scavenging: A Non-disruptlve High Perfornm.nce Storage Reclamation Algorithm 1. Introduction 2. The Relationship BetwSend us Fan Mail

This week we present the first of two episodes on dynamic memory allocation.  We were going to talk about garbage collection, but realized we should probably discuss allocators first.  Turns out they are complicated, and there are many different strategies.Dynamic Storage Allocation: A Survey and Critical Review  (Wilson)Hoard: a scalable memory allocator for multithreaded applicationsmimalloc: A new, high-performance, scalable memory allocator for the modern era - Microsoft ResearchA Scalable Concurrent malloc(3) Implementation for FreeBSDSend us Fan Mail

In this episode we interview Alan Dyke, a colleague of Erik's who got his Ph.D from the University of Wales, Aberystwyth, where he did research on Computer-Aided Software Engineering.  We talk about Alan's unique journey in software development, going from the academic environment to the defense industry, embedded software, and biotech.  We also learn about the crucial importance of calibrating your software for hurricanes.Send us Fan Mail

This week we're talking about  Mike's visit to DataCon LA 26.  We talk about the conference, various talks he was able to see, and his own experience giving a talk.Compression Represents Intelligence Linearly[2602.17127] The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AIInstructorspaCyJohn Snow LabsHierarchical Navigable Small Worlds (HNSW) | PineconeLance Video is a demonstration platform for analyzing and searching video content with LanceDB. · GitHubSend us Fan Mail

In this episode we discuss the Monte Carlo method, both the technique and its place in the history of computing.  The Monte Carlo method is a set of tools that use probability to solve problems that are otherwise intractable.  Programs using Monte Carlo were among the first programs created on digital computers.The Monte Carlo Method Nicholas Metropolis; S. Ulam Journal of the American Statistical Association, Vol. 44, No. 247. (Sep.Equation of State Calculations by Fast Computing MachinesHitting the Jackpot: The Birth of the Monte Carlo Method | LANLMonte Carlo Sampling Methods Using Markov Chains and Their Applications W. K. Hastings Biometrika, Vol. 57, No. 1. (Apr., 1970),2014HaighPriestleyRope-LosAlamosBetsonENIAC.pdfThe Unheralded Contributions of Klara Dan von NeumannSend us Fan Mail

In this episode we discuss the 5 most-cited papers from the flagship journal of the Association for Computing Machinery, the JACM.   The papers include everything from distributed computing to real-time scheduling. Direct Search'' Solution of Numerical and Statistical Problems | Journal of the ACM Impossibility of distributed consensus with one faulty process | Journal of the ACM Robust principal component analysis? | Journal of the ACM Authoritative sources in a hyperlinked environment | Journal of the ACM Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment | Journal of the ACMWhat is HITS Algorithm (Hyperlink-Induced Topic Search)?Send us Fan Mail

This episode started with the question: who first had the idea to connect computers together to make a network?  We explore JCR Licklider's ideas about an  "Intergalactic Network", some of the earliest attempts at networking, and the ideas that lead to packet switching and the Arpanet.SAGE: Semi-Automatic Ground Environment Air Defense System | MIT Lincoln LaboratoryMemorandum for Members and Affiliates of the Intergalactic Computer NetworkThe Arpanet and Computer Networks60 Years of Networking - by Bruce Davies - Systems ApproachInformation Flow in Large Communication Nets (1961, Leonard Kleinrock, MIT)On Distributed Communications Networks (1962, Paul Baran, RAND)National Physical Laboratory Proposal for a Digital Communication Network by D. W. Davies June, 1966Send us Fan Mail

In this episode we discuss a couple of recent papers on just-in-time development and agentic systems and how we think they suggest trends that will affect software development for years to come.  We boldly predict that in 2046 people will still not want robot dentists.The Time is Here for Just-in-Time Systems: Challenges and OpportunitiesMDASH: The MS Security Thing Services: The New Software | Sequoia CapitalUnderstanding Spec-Driven-Development: Kiro, spec-kit, and TesslSend us Fan Mail