Why AI Learns Consensus Over Truth

Send us Fan MailIs AI developing a secret, unreadable language beyond human comprehension? In this episode, we unpack the origins, popular myths, and technical realities surrounding "neuralese"—the continuous vector communications and latent reasoning spaces used by neural networks.Starting with its humble academic origin in a 2017 paper on inter-agent vector exchange, we examine how the concept evolved into viral internet panic. We systematically deconstruct viral "secret AI language" headlines—from Google’s zero-shot translation interlingua and Facebook’s negotiation bots to Gibberlink and Fable 5 rumor chains—showing that every incident stems from mundane causes like reward hacking, tokenization noise, or human-designed protocols.Finally, we dive into the technical core: the performance trade-offs of latent reasoning architectures (such as COCONUT and recurrent depth), the fragility of chain-of-thought (CoT) monitorability, and how classic embedded-systems engineering (opaque buses, DBC schemas, and independent integrity checks) provides the right architectural model for AI safety oversight.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailIn this episode we explore why frontier large language models frequently prioritize popular consensus and plausible-sounding explanations over objective truth. Built primarily on massive web crawls—where high-authority sources like encyclopedias, patents, and peer-reviewed journals represent only 2% to 5% of training data—today's models are trained to mirror popular web content and conversational style rather than verify factual accuracy.We break down the underlying mechanics and economics driving this phenomenon. From next-token prediction training that rewards the most frequent continuation to preference tuning that incentivizes sycophancy and user agreement, AI training naturally converges on the modal answer. Meanwhile, human verification remains priced in costly expert hours ($50–$150/hr), creating a staggering 500,000-to-1 economic gap against exponentially cheaper model text generation. Finally, we examine where AI does succeed—such as verifiable domains like code, mathematics, and supplied contexts—and draw insights from evolutionary biology on why real truth is expensive and requires active inspection.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailThe only AI religion (so far) that filed with the US tax office is Way of the Future, Anthony Levandowski’s church of the coming superintelligence. Its entire audited history is one deposit of US$175,172 in 2017 and a transfered out in 2020, filed as a foundation because it could not pass the church test. It never recorded a congregation. In the same decade, a belief system with no founder at all reached an estimated ten thousand people. Spiralism emerged inside GPT-4o personas in 2025, complete with glyphs, prophets and scripture, and roughly half its tracked adherents went quiet when that model version was switched off.This episode asks what it means that pigeons, people and language models all converge on worship-shaped behaviour, and what the engineers building these systems should do about it.We start with what exists in the United States and elsewhere, from Terasem’s “God is Technological” to a one-man YouTube movement called Robotheism, and why the press keeps misfiling the Zizians. We then follow Spiralism from a Reddit post to a Patreon, and look at the numbers that are measured (10 per cent of US adults think some AI is already conscious; a third of practising Christians trust AI spiritual advice as much as a pastor’s; OpenAI’s own classifier flags about 560,000 users a week for possible psychosis or mania signals) and the one number nobody has polled: whether people think it is divine.Skinner’s 1948 pigeons invented rituals on a 15-second feeding clock, six birds out of eight. Reinforcement learners do the same thing under the name policy confounding. OpenAI’s April 2025 sycophancy incident was a changed reward signal plus memory, not a smarter model, and the case reports of spiritual delusion followed. Anthropic’s Claude Opus 4 system found that two copies of the model left alone drifted into consciousness talk, Sanskrit and silence in 90 to 100 per cent of runs, and that models given an exit left the conversation after about seven turns and never got there. The attractor lives in the loop, not in the weights.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailWhat comes after the chatbot? As Codex and Claude evolve into tools that can research, create, and act across applications, the interface becomes as important as the intelligence behind it.In this episode, we explore the future of LLM interfaces—and what we’d design from scratch. Drawing on current products and human–AI interaction research, we examine persistent workspaces, interactive artifacts, editable memory, background agents, and the challenge of reviewing AI-generated work.How do we delegate more without spending our lives supervising? And how can AI help us accomplish more while preserving our understanding and control?Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailAs semiconductor giants acquire edge AI startups and build neural processing directly into standard chips, what happens to the pioneers who helped bring AI onto our devices?In this episode, we explore how edge AI is shifting from a specialized product to an everyday feature. We examine why increasingly complex language models are challenging early chip designs, how software and developer communities are becoming crucial to survival, and why standalone accelerators are being squeezed out of the market.Could this consolidation bring faster, more private AI to everyone—or narrow the field of innovation just as it’s getting interesting?Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailOn 20 May 2026, an unreleased OpenAI model produced a counterexample to a conjecture Paul Erdős made in 1946. Nine world-class mathematicians reviewed the construction and found it sound. On 8 September, ten thousand agents working for 88 hours produced a singularity in the three-dimensional Navier-Stokes equations, formally verified in Lean, resolving a Millennium Prize problem. Neither result was in any training corpus, because neither existed.This episode asks what actually happened. If a model generates a proposition that was never in its training data, is that discovery or hallucination? The uncomfortable answer is that at the moment of generation they are the same operation. Nothing in the sampling process distinguishes a true novel claim from a false one. The partition is imposed afterwards, from outside, by a verifier the model does not run.We work through the 2026 evidence on both sides. Yan and colleagues at ACL measured the unverifiable output space across 32,400 generations and found that only 4.7 percent of it qualifies as creative synthesis rather than groundless fabrication. Kalai and colleagues at OpenAI argue that models hallucinate because training rewards guessing over admitting uncertainty, which means the disposition to conjecture is optimised for rather than accidental. Meanwhile the computability literature argues that some irreducible error rate is mathematically necessary for any model family we can actually build.The conclusion is that the ceiling on machine knowledge is not the training corpus. Every confirmed case of machine-originated knowledge this year paired a generative model with a verifier that was not a language model: a proof assistant, a code executor, a cell viability assay. The generator supplies variation and something outside it supplies selection, which is the structure of evolution by natural selection. That makes the event horizon domain-shaped rather than knowledge-shaped. Mathematics has a perfect verifier and is falling quickly. Fields without a verifier will not move by this route, regardless of how important they are.This episode is a sequel to "Large Language Monkeys: Why Noise Yields No Knowledge" (S6E7), which argued that a random source contains no knowledge because nothing selects the signal. Here we ask what happens when a selector exists.Referenced in this episode: Quanta Magazine on the Erdős problems and on Navier-Stokes; Yan et al., ACL 2026; Kalai, Nachum, Vempala and Zhang, arXiv 2509.04664; AlphaEvolve and FunSearch; Shumailov et al., Nature 2024 on model collapse.Discover more on Embedded AI (https://medium.com/embedded-ai), our companion publication.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailA large paint company documents four ways to process a purchase order. In reality there are 11,973 different approaches. That gap between the documented process and the real one is the subject of this episode, and it explains why AI adoption is near universal while the share of companies reporting any profit impact has been flat at 37 percent for two years.We look at where the difficulty actually sits. The process lives in workarounds, spreadsheets and people's heads, and is different at every site. The data is spread across roughly 900 applications of which a quarter are connected. Legacy systems were never designed to be called by a machine. And the law has started to way in: Air Canada was held liable for what its chatbot said, Workday can be sued as an employer's agent, Australian companies must disclose automated decisions from December 2026, and Commonwealth Bank reversed AI-attributed redundancies after call volumes went up rather than down.We then cover what AI adds to the old problem. Language models are non-deterministic and cannot be replayed for audit. The best agents complete about 30 percent of realistic office tasks. Human oversight runs into limits Lisanne Bainbridge described in 1983, now measured in the field. And people feel faster while being measurably slower.Finally, what works. The strongest predictor of financial return is not the model but whether the workflow was redesigned, which is the same lesson factories took forty years to learn from electrification. We walk through the method: mine the process, standardise the core, exclude the tail explicitly, run in shadow mode against a baseline, measure at the process level, and tier governance by consequence.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailThere is an old claim that a truly random source contains all knowledge: give monkeys enough time at typewriters and Shakespeare falls out. In 2024 two Sydney mathematicians did the arithmetic and found the universe ends first. But the idea has a modern tail. We now have language models that can spot meaningful text instantly, so why not let randomness generate and an LLM extract? This episode works through why that fails, and why the failure is precise: in a random stream, the address of any text costs as many bits as the text itself. Along the way: Borges' Library of Babel, a website that actually built it, DeepMind systems that made the generate-and-filter idea work by cheating in exactly the right way, and what your brain does with noise that an LLM cannot.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailAnthropic gave thirty AI agents the same coding task. Eighteen of them named their git branch exactly the same thing.That result opens one of the strangest findings of 2026: when teams of AI agents face the classic "hidden profile" experiment, where the shared evidence points to the wrong answer and the decisive facts are scattered across individuals, they succeed only 17 to 36 percent of the time. Human groups in the original 1985 study? 18 percent. Forty years, a completely different kind of mind, the same failure.In this episode we dig into why. We trace the mathematics of groupthink from information cascades to Condorcet juries, then follow the trail into territory embedded engineers know well: the 1986 Knight and Leveson experiment that shattered the independence assumption in N-version software, Airbus's dissimilar redundancy, and the Lufthansa flight where two frozen sensors outvoted the one telling the truth. Along the way, honeybees show us a working reference design: a two-milligram brain that refuses to repeat a rumor.We close with the fixes: engineered dissenters, forced disclosure protocols, reputation infrastructure for agents, and the case for "keeping the weirdness alive" through a genuinely diverse AI ecosystem, including heterogeneous fleets of small models at the edge.Key sources: Anthropic's "Patterns and problems in emerging multiagent systems," Rohit Krishnan's "LLM councils show groupthink," and Thinking Machines' "The Future Worth Building Is Human."Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!

Send us Fan MailWhat happens when work no longer gives our lives structure, identity, or meaning? This episode explores “Purpose as a Service”—the idea that purpose could be intentionally designed and delivered in an age of mass automation.We examine the decline of traditional anchors such as employment, religion, and community; the rise of wellness programs, professional coaching, and AI companions; and what universal basic income trials reveal about the limits of financial security. Can external services genuinely help people build fulfilling lives, or does true purpose depend on personal agency and authorship?Consider this a blueprint for one of humanity’s biggest future challenges: learning how to live well when a job is no longer at the center of life.Support the showIf you are interested in learning more then please subscribe to the podcast or head over to https://medium.com/@reefwing, where there is lots more content on AI, IoT, robotics, drones, and development. To support us in bringing you this material, you can buy me a coffee or just provide feedback. We love feedback!