
Why AI Benchmarks Are Broken, Plus Turning Down $5M | Jocelyn D'Arcy
19/11/2026
0:00
1:14:29
In this episode of Nothing Ventured, Aarish Shah sits down with Jocelyn D'Arcy, founder of Poindexter Labs, to explore AI benchmarks, high-quality training data and what venture funding really buys a profitable founder.
A former maths teacher, Jocelyn built Poindexter Labs after doing Maths Olympiad annotation work, and she is clear that what the company does is not data labelling. Instead of sorting existing data, her team starts from a blank page and creates problems at the edge of what large language models can do. She explains why benchmarks can be like exam boards selling the textbook for the test, and how wrong answers and under-specified questions skew the way AI models are judged.
The conversation then turns to the business. Jocelyn describes a coalition of expert contributors, "Poindexters", who are vetted through live subject knowledge interviews. She explains why Poindexter Labs, already profitable, took venture funding to earn credibility with frontier labs, and why she turned down a $5 million subcontract.
In this episode, we cover:
- Why AI benchmarks need independent oversight, and the exam board analogy
- Why Poindexter Labs is "not data labelling"
- Why LLMs avoid tool calls, and how post-training with RL and SFT helps
- How to vet and retain expert contributors so review chains stop rejecting good work
- AI adoption in SMEs, "AI employees" and how much autonomy agents should have
- Ads in ChatGPT, GEO and the changing face of search
- Turning down $5 million of revenue and what venture capital bought a profitable startup
Find Jocelyn D'Arcy online at:
LinkedIn: https://www.linkedin.com/in/jocelyndarcy/
Twitter: https://x.com/Jocelyn_K_DArcy
Instagram: https://www.instagram.com/poindexterlabs/?hl=en
Company: https://poindexterlabs.ai
This episode is sponsored by EmergeOne, fractional CFOs for venture backed tech startups from Seed to Series B.
Subscribe to Nothing Ventured for more conversations with the founders, investors and innovators shaping the future of business and technology.
#NothingVentured #AI #LLMs #AIBenchmarks #DataLabelling #VentureCapital #Startups #Founders #FrontierLabs #PoindexterLabs Hosted by Simplecast, an AdsWizz company. See https://pcm.adswizz.com
for information about our collection and use of personal data for
advertising.
D'autres épisodes de "Nothing Ventured"



Ne ratez aucun épisode de “Nothing Ventured” et abonnez-vous gratuitement à ce podcast dans l'application GetPodcast.








