The Coding Interview Is Dead: What Do We Test Now?

In its amended IPO filing, Anthropic told investors it may soon reach its own threshold for AI-driven acceleration of AI research. Three days later, the independent evaluator METR found that progress has likely doubled, but said most of its key evidence was redacted by Anthropic. Roger Basler de Roca and his co-host Sunday ask what transparency is worth when the examined party decides what the public may see.In this episode:- What Anthropic disclosed in its SEC filing, and what METR measured: around twice the pace of improvement on well-specified tasks- Why an assessment with unilateral redaction rights looks like an audit but is not yet verification- What business leaders should change: separate vendor claims from evidence, shorten AI review cycles, keep a human who can say noSources:Anthropic. (2026, October 26). Anthropic files S-1 with the SEC for proposed initial public offering. https://www.anthropic.com/news/s1-secBort, J. (2026, November 2). Anthropic warns that it's very close to RSI capability, aka self-building AI. TechCrunch. https://techcrunch.com/2026/11/02/anthropic-warns-that-its-very-close-to-rsi-capability-aka-self-building-ai/Gizmodo. (2026, November). Anthropic claims to be near recursive self-improvement. https://gizmodo.com/anthropic-claims-to-be-closing-in-on-recursive-self-improvement-2000834175METR. (2026, November 5). Independent assessment of AI R&D acceleration at Anthropic in October 2026. https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/More from Roger Basler de Roca: https://thinkroger.aiLinkedIn: https://www.linkedin.com/in/rogerbaslerFeedback: [email protected] Sunday is produced with AI: Roger's voice is his own ElevenLabs voice clone, his co-host Sunday is an AI voice. Research and script are AI-assisted and reviewed for facts.

An engineering manager at Anthropic says he has stopped using coding interviews, because Claude now solves the small challenges better than candidates do. Roger Basler de Roca and his co-host Sunday ask what that means for hiring, for juniors, and for every job whose entry test assumed that producing the output proves the skill.In this episode:Why a test stops working once a tool can pass it, and the two opposite reactions: drop the test, or lock the doorThe risk behind "hunger beats tenure": faster teams, weaker foundations, and fewer people who can check the machineThree things leaders can change now: audit their own tests, assess judgment around AI output, and rebuild apprenticeship for juniorsSources:Claburn, T. (2026, October 28). Anthropic engineer declares the coding interview dead. The Register. https://www.theregister.com/ai-and-ml/2026/10/28/anthropic-engineer-declares-the-coding-interview-dead/5306115Rieseberg, F. (2026, October 27). Hunger beats tenure: Advice for junior engineers. https://felixrieseberg.com/hunger-beats-tenure-advice-for-junior-engineers/The Decoder. (2026). With AI writing the code, entrepreneurial thinking is now what counts most for newcomers at Anthropic. https://the-decoder.com/with-ai-writing-the-code-entrepreneurial-thinking-is-now-what-counts-most-for-newcomers-at-anthropic/Bachaud, N. (2026, October 26). Better work, more to learn: The 2026 AI worker report. ZipRecruiter. https://www.ziprecruiter-research.org/economic-insights-research/2026-ai-worker-reportMore from Roger Basler de Roca: https://thinkroger.aiLinkedIn: https://www.linkedin.com/in/rogerbaslerFeedback: [email protected] Sunday is produced with AI: Roger's voice is his own ElevenLabs voice clone, his co-host Sunday is an AI voice. Research and script are AI-assisted and reviewed for facts.

Dartmouth's provost resigned after a faculty committee found he had substantially altered his published writing with AI without adequate disclosure. In this first episode of Hello Sunday, Roger Basler de Roca and his co-host Sunday ask where AI assistance ends and substitution begins, and what leaders owe the people who read their words.In this episode:- What happened at Dartmouth, and why the AI detector started the story but the evidence ended it- Ghostwriter vs. language model: why AI-polished text is not the same as a speechwriter's draft- Three rules for leaders: disclose like you require others to, separate assistance from substitution, keep the thinkingSources:Esaki-Smith, A. (2026, October 30). Dartmouth provost steps down amid AI controversy. Forbes. https://www.forbes.com/sites/annaesakismith/2026/10/30/dartmouth-provost-steps-down-amid-ai-controversy/Palmer, K. (2026, September 29). Dartmouth provost's AI use sparks backlash. Inside Higher Ed. https://www.insidehighered.com/node/220201Shibu, S. (2026, September 30). Dartmouth professor used AI to write an article about the dangers of AI. Entrepreneur. https://www.entrepreneur.com/business-news/dartmouth-professor-used-ai-to-write-an-article-about-the-dangers-of-ai-hypocrisyThe New York Times. (2026, September 29). College leaders are using AI. Students call it hypocrisy. Boston.com. https://www.boston.com/news/education/2026/09/29/college-leaders-are-using-ai-students-call-it-hypocrisy/More from Roger Basler de Roca: https://thinkroger.aiLinkedIn: https://www.linkedin.com/in/rogerbaslerFeedback: [email protected] Sunday is produced with AI: Roger's voice is his own ElevenLabs voice clone, his co-host Sunday is an AI voice. Research and script are AI-assisted and reviewed for facts.

In this episode, we enter the world of Large Reasoning Models (LRMs). We explore advanced AI systems such as OpenAI’s o1/o3, DeepSeek-R1, and Claude 3.7 Sonnet Thinking—models that generate detailed "thinking processes" (Chain-of-Thought, CoT) with built-in self-reflection before answering. These systems promise a new era of problem-solving. Yet, their true capabilities, scaling behavior, and limitations remain only partially understood.By conducting systematic investigations in controlled puzzle environments—including the Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World—we uncover both the strengths and surprising weaknesses of LRMs. These environments allow precise control over task complexity while avoiding data contamination issues that often plague established benchmarks in mathematics and coding.A striking finding: LRMs face a complete accuracy collapse beyond certain complexity thresholds. Paradoxically, their reasoning effort (measured in "thinking tokens") first increases with complexity, only to decline after a point—even when token budgets are sufficient.We identify three distinct performance regimes:Low-complexity tasks – where standard Large Language Models (LLMs) still outperform LRMs.Medium-complexity tasks – where LRMs’ additional "thinking" shows a clear advantage.High-complexity tasks – where both LLMs and LRMs collapse entirely.Another challenge is “overthinking.” On simpler problems, LRMs often find correct solutions early but continue to pursue false alternatives, wasting computational resources. Even more surprising is their weakness in exact computation: they fail to leverage explicit algorithms, even when provided, and show inconsistent reasoning across different puzzle types.This episode invites you to rethink assumptions about AI’s capacity for generalizable reasoning. What does it truly mean for a machine to "think" under increasing complexity? And how should these insights shape the next generation of AI design and deployment? Sources: Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025). The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. (Unpublished manuscript). https://arxiv.org/abs/2506.06941 Disclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

Join us as we dive into a groundbreaking study that systematically investigates the strengths and fundamental limitations of Large Reasoning Models (LRMs), the cutting-edge AI systems behind advanced "thinking" mechanisms like Chain-of-Thought with self-reflection.Moving beyond traditional, often contaminated, mathematical and coding benchmarks, this research uses controllable puzzle environments like the Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World to precisely manipulate problem complexity and offer unprecedented insights into how LRMs "think".You'll discover surprising findings, including:Three distinct performance regimes: Standard Large Language Models (LLMs) surprisingly outperform LRMs on low-complexity tasks; LRMs demonstrate an advantage on medium-complexity tasks due to their additional "thinking" processes; but crucially, both model types experience a complete accuracy collapse on high-complexity tasks.A counter-intuitive scaling limit: LRMs' reasoning effort, measured by token usage, increases up to a certain complexity point, then paradoxically declines despite having an adequate token budget.This suggests a fundamental inference-time scaling limitation in their reasoning capabilities relative to problem complexity.Inconsistencies and limitations in exact computation: LRMs struggle to benefit from being explicitly given algorithms, failing to improve performance even when provided with step-by-step instructions for puzzles like the Tower of HanoiThey also exhibit inconsistent reasoning across different puzzle types, performing many correct moves in one scenario (e.g., Tower of Hanoi) but failing much earlier in another (e.g., River Crossing), indicating potential issues with generalizable reasoning rather than just problem-solving strategy discovery"Overthinking" phenomenon: For simpler problems, LRMs often find correct solutions early in their reasoning trace but then continue to inefficiently explore incorrect alternatives, wasting computational effortThis episode challenges prevailing assumptions about LRM capabilities and raises crucial questions about their true reasoning potential, paving the way for future investigations into more robust AI reasoning.Disclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

In this show, we break down the art of crafting prompts that help AI deliver precise, useful, and reliable results.Whether you're summarising text, answering questions, generating code, or translating content — we’ll show you how to guide LLMs effectively.We explore real-world techniques, from simple zero-shot prompts to advanced strategies like Chain of Thought, Tree of Thoughts, and ReAct, combining reasoning with external tools.We’ll also dive into how to control AI output — tweaking things like temperature, token limits, and sampling settings — to shape your results.Plus, we’ll share best practices for writing, testing, and refining prompts — including tips on examples, formatting, and structured outputs like JSON.Whether you’re just getting started or already deep into advanced prompting, this podcast will help you sharpen your skills and stay ahead of the curve.Let’s unlock the full potential of AI — one prompt at a time.Disclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

Has AI finally passed the Turing Test? Dive into the groundbreaking news from UC San Diego, where research published in March 2025 claims that GPT 4.5 convinced human judges it was a real person 73% of the time, even more often than actual humans in the same test. But what does this historic moment truly signify for the future of artificial intelligence?This podcast explores the original concept of the Turing Test, proposed by Alan Turing in 1950 as a practical measure of a machine's ability to exhibit intelligent behavior indistinguishable from that of a human through conversation. We'll examine the rigorous controlled study that led to GPT 4.5's alleged success, involving 284 participants and five-minute conversations.We'll delve into what passing the Turing Test actually means – and, crucially, what it doesn't. Is this the dawn of true AI consciousness or Artificial General Intelligence (AGI)? The sources clarify that the Turing Test specifically measures conversational ability and human likeness in dialogue, not sentience or general intelligence.Discover the key factors that contributed to this breakthrough, including massive increases in model parameters and training data, sophisticated prompting (especially the use of a "persona prompt"), learning from human feedback, and models designed for conversation. We will also discuss the intriguing finding that human judges often identified someone as human when they lacked knowledge or made mistakes, showing a shift in our perception of AI.However, the podcast will also address the criticisms and limitations of the Turing Test. We'll explore the argument that it's merely a test of functionality and doesn't necessarily indicate genuine human-like thinking. We'll also touch on alternative tests for AI that aim to assess creativity, problem-solving, and other aspects of intelligence beyond conversation, such as the Metzinger Test and the Lovelace 2.0 Test.Finally, we will consider the profound implications of AI systems convincingly simulating human conversation, including the economic impact on roles requiring human-like interaction, the potential effects on social relationships, and the ethical considerations around deception and manipulation.Join us to unpack this milestone in computing history and discuss what the blurring lines between human and machine communication mean for our society, economy, and lives.Source: https://theconversation.com/chatgpt-just-passed-the-turing-test-but-that-doesnt-mean-ai-is-now-as-smart-as-humans-253946Disclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

h 145-page paper from Google DeepMind, outlining their strategic approach to managing the risks and responsibilities of AGI development.1. Defining AGI and ‘Exceptional AGI’We begin by clarifying what DeepMind means by AGI: an AI system capable of performing any task a human can. More specifically, they introduce the notion of ‘Exceptional AGI’ – a system whose performance matches or exceeds that of the top 1% of professionals across a wide range of non-physical tasks.(Note: DeepMind is a British AI company, founded in 2012 and acquired by Google in 2014.)2. Understanding the Risk LandscapeAGI, while full of potential, also presents serious risks – from systemic harm to outright existential threats. DeepMind identifies four core areas of concern:Abuse (intentional misuse by actors with harmful intent)Misconduct (reckless or unethical use)Errors (unexpected failures or flaws in design)Structural risks (long-term unintended societal or economic consequences)Among these, abuse and misconduct are given particular attention due to their immediacy and severity.3. Mitigating AGI Threats: DeepMind’s Technical StrategyTo counter these dangers, DeepMind proposes a multi-layered technical safety strategy. The goal is twofold:To prevent access to powerful capabilities by bad actorsTo better understand and predict AI behaviour as systems grow in autonomy and complexityThis approach integrates mechanisms for oversight, constraint, and continual evaluation.4. Debate Within the AI FieldHowever, the path is far from settled. Within the AI research community, there is ongoing skepticism regarding both the feasibility of AGI and the assumptions underlying safety interventions. Critics argue that AGI remains too vaguely defined to justify such extensive safeguards, while others warn that dismissing risks could be equally shortsighted.5. Timelines and TrajectoriesWhen might we see AGI? DeepMind’s report considers the emergence of ‘Exceptional AGI’ as plausible before the end of this decade – that is, before 2030. While no exact date is predicted, the implication is clear: preparation cannot wait.This episode offers a rare look behind the scenes at how a leading AI lab is thinking about, and preparing for, the future of artificial general intelligence. It also raises the broader question: how should societies respond when technology begins to exceed traditional human limits? Source: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/evaluating-potential-cybersecurity-threats-of-advanced-ai/An_Approach_to_Technical_AGI_Safety_Apr_2025.pdfDisclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

This academic paper from Anthropic provides an empirical analysis of how artificial intelligence, specifically their Claude model, is being used across the economy. The researchers developed a novel method to analyse millions of Claude conversations and map them to tasks and occupations listed in the US Department of Labor's O*NET database. Their findings indicate that AI usage is currently concentrated in areas like software development and writing, with a notable portion of occupations showing AI use for some of their tasks. The study also distinguishes between AI being used to automate tasks versus augment human capabilities and examines usage patterns across different Claude models, providing early, data-driven insights into AI's evolving role in the labour market.Source: https://www.anthropic.com/news/the-anthropic-economic-indexDisclaimer: This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/

A study by the Columbia Journalism Review investigated the ability of eight AI search engines to accurately cite news sources. The findings revealed significant shortcomings across all tested platforms, including a tendency to provide incorrect information with unwarranted confidence and fabricate citations or link to incorrect versions of articles. Premium AI models were found to offer more confidently inaccurate answers than their free counterparts. Furthermore, several chatbots appeared to disregard publishers' instructions in their robots.txt files, and content licensing agreements did not guarantee accurate sourcing. Overall, the research highlights a widespread problem with AI search engines struggling to properly attribute and link to original news content, potentially harming both publishers and users.Source: https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.phpDisclaimer: This podcast is generated by Roger Basler de Roca (contact) with the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.⁠https://rogerbasler.ch/en/contact/