S8 Ep34: How good are LLMs at doing our jobs?

 In the second of special series recorded live at the PSE-CEPR Policy Forum 2025, we are asking, how good is AI at doing real-world job task? And how can we measure their capability without resorting to technical benchmarks that may not mean much in the workplace? Since we all became aware of large language models, LLMs scientists have been attempting to evaluate how good they are at performing expert tasks. The results of those tests can show us whether LLMs  can be useful complements to our work, or even replacements for us, as many fear. But setting or grading a test to decide whether an LLM can do a problem-solving job task, rather than solve an abstract problem, isn't easy to do.  Maria del Rio-Chanona, a computer scientist at UCL, tells Tim Phillips about her innovative work-in-progress, in which she asks an LLM to set a tricky workplace exam, then tells another LLM to take the test – which a third LLM evaluates.

Recorded live at the PSE-CEPR Policy Forum 2025.  This year the annual Paris School of Economics-PSE Policy Forum is organized around three themes: artificial intelligence and labour reallocation, working conditions and remote work, and inequality in the workplace. In short, what's work going to look like in the future?  Our series of podcasts, recorded live at the event, starts with David Autor’s work on the impact of AI on jobs. Rather than speculate about how soon AI will destroy work, David’s research focuses on which tasks AI will automate, and what that means in terms of the expertise needed to do these jobs in the future. He tells Tim Phillips that some jobs will become more expert and some less – but the employment effects of AI may be the opposite of what many people expect. 

The Bank for International Settlements Annual Economic Report has just dropped, and there’s a markedly less positive tone than last year, when it was celebrating imminent soft landings in the global economy. It warns of a deteriorating outlook for growth, coupled with vulnerabilities in the global financial system.  So, what exactly is the BIS worried about, how can policy and regulation respond, and should central banks start worrying about the next systemic crisis? Gaston Gelos and Frank Smets are Deputy Heads of the Monetary and Economic Department at the BIS and are also two of the authors who put together the report. They talk to Tim Phillips about why last year’s optimism has disappeared, and how monetary and fiscal policy can adjust to cope with a new era of uncertainty and fragmentation. 

Whether you are looking at the link from education to economic growth, household earnings or individual happiness, there’s no doubt that a better-educated population is good news. But how can policy improve education in a cost-effective way? You might assume that a good route would be to improve the management of schools, but existing research is not conclusive, and often top-down attempts to improve management meets opposition from administrators. An experiment in Brazil has evaluated a program to improve management using existing resources in Rio de Janeiro. Tiago Cavalcanti, of University of Cambridge, Sao Paulo School of Economics & CEPR, and Felipe Puccioni of the Court of Accounts of Rio de Janeiro came up with the experiment, and they tell Tim Phillips about why universal education doesn’t necessarily mean universal learning – and how they became celebrities on national television when their successful project hit the headlines.

Pedestrianised areas, car-free streets, or low traffic neighbourhoods are increasingly visible in major cities. Whether in London, Paris, New York or Barcelona, these changes are always controversial – but does the loud criticism that we often hear in social media or newspapers really represent the views of voters who are affected by these policies? Elisabet Viladecans-Marsal of the Universitat de Barcelona, Barcelona Institute of Economics and CEPR spoke to Tim Phillips about whether Barcelona’s car-free “Superblocks” were vote-winners or vote-losers for the city’s mayor. Photo: Cataleirxs

What’s the point of having a job? Clearly, to make money for ourselves and our families. But is it possible for us to discover some bigger purpose or meaning at work. And, if we do, who benefits? That’s the idea that a multinational organisation had when it called in a team of economists to analyse its internal programme called “Find your Purpose” (FYP). The resulting RCT set out to measure whether FYP changed how employees behaved at work, whether it helped them enjoy their jobs, and whether it increased profits too. Oriana Bandiera of London School of Economics and CEPR was one of those economists. She tells Tim Phillips how she took the programme and found her purpose, why FYP increased the quit rate but improved productivity, and why employees who took the programme stopped worrying about their work-life balance. 

Does the public take more notice of the opinions of male or female economists? We know that female experts, whether in science, politics or the media, suffer from an authority gap: their expertise is often not given as much weight by the public as opinions held by less qualified men. But does the gap persist for the very highest achievers? And, if it closes or even reverses for them, what lessons are there for other female experts?  Sarah Smith of the University of Bristol and CEPR recently conducted an experiment about which expert economists are most likely to influence public opinion. She tells Tim Phillips about a signal that reverses the authority gap, and how this insight can help other female economists to communicate their expertise. 

What if trade policy wasn’t really about trade at all? What if it was about revenge, power, and punishment, tariffs as tantrums and diplomacy as drama? You won’t find the Grievance Doctrine in economics textbooks, but there is one book that explains what it is, what its policies are, and the way it is currently being implemented. Richard Baldwin of IMD Business School in Lausanne, the founder and the Editor-in-Chief of VoxEU is also the author of “The Great Trade Hack”. In it, he sets out the way the Grievance Doctrine has been weaponised by this US administration, how the rest of the world could respond, and what might happen next. Richard joins Tim Phillips to explain the thinking that guides policy one of the most extraordinary periods in the history of trade – and why the rest of world will do just fine without the US as an ally. Download The Great Trade Hack.

China’s growth as an economic superpower has been based in a large part on its increasing ability to design and manufacture sophisticated, hi-tech goods. But, until recently, it was far from a superpower when it came to creating new knowledge and cutting-edge academic research. Luc Laeven of the ECB and CEPR and his co-authors recently published an analysis of the research output in top journals from Chinese academics over the last two decades, and the results are startling: in many areas of science, China is now clearly the world leader. Luc talks to Tim Phillips about how China’s plan was created, why the quality as well as the quantity of research should make us take notice, and whether research establishments in Europe and the US can learn from China’s single-minded pursuit of success.

Does economic growth inspire us to trust our governments? A new paper finds a surprisingly strong and consistent relationship between trust and economic growth – not for this quarter, or this year, but over our lifetimes. Tim Besley of the London School of Economics tells Tim Phillips how we can measure trust in a government around the world, and the strong and consistent relationship between long-run growth and trust.