Data & AI Mastery podcast

Why a 1B Model Beat a 70B LLM at Following the Rules

0:00
1:00:20
Manda indietro di 15 secondi
Manda avanti di 15 secondi

👉 Discover how Cambridge Spark helps organisations build the data and AI capabilities needed to turn strategy into measurable impact: [cambridgespark.com](https://www.cambridgespark.com/?utm_campaign=6955829-Podcast%202024%20-%20Onwards&utm_source=Podcast&utm_medium=RSS)


An LLM can give well-formed output 99% of the time and still break the domain's rules in most of it.

You want to put an LLM into a real decision process, such as invoice approvals, pricing or clinical recommendations. The answers read well. But are they actually allowed, and do they solve the problem?

Professor Alessandra Russo is Professor in Applied Computational Logic at Imperial College London and Director of the Thomson Reuters Imperial Frontier AI Research Lab. She co-directs the UKRI Centre for Doctoral Training in Safe and Trusted AI with King's College London. Her neuro-symbolic AI research combines logic with machine learning to build AI systems with functional guarantees.

Jeremy Bradley and Alessandra go inside SEM-CTRL, her team's method for semantically controlled decoding in open weight models. You'll hear how a symbolic checker sits in the decoder and masks invalid tokens before they are ever chosen. Monte Carlo tree search then steers generation towards correct answers. No fine-tuning required.

This episode is for data scientists, ML engineers and technical leaders responsible for enterprise AI deployment in regulated or high-stakes settings. It assumes some comfort with how LLMs generate tokens. If you want a light overview of AI trends, this one goes further.


## Chapter Markers

00:00 Why AI should be able to say "I don't know"

00:19 Introducing Professor Alessandra Russo

01:32 What organisations worry about beyond hallucination

04:30 Open weight models versus frontier APIs

08:37 The risks that come with open weights

11:16 Inside the SE-CTRL paper

12:39 Validity versus correctness in the blocks world

16:46 A pricing guardrail that needs memory

19:48 Context-sensitive grammars and answer set grammars

24:39 Where the checker sits in the decoder

28:00 Masking tokens before softmax, and temperature

30:20 Monte Carlo tree search for correct solutions

39:01 Where reward functions come from

40:18 Why a 1B model outperformed a 70B model

46:29 Who writes the grammar: ILASP and LLMs

52:47 Limits: fairness, noise and robotics

55:23 What an NHS trust should do first

59:07 Where to find the research


## Useful Links & Resources

- SEM-CTRL: Semantically Controlled Decoding: https://doi.org/10.48550/arXiv.2503.01804

How LLMs and agents can be used as robust decision agents in complex decision problems, where hard safeguards can be put in place to prevent unsafe outcomes and reinforcement learning can be used to iterate towards better solutions.

- Answer Set Grammars (AAAI 2019)

- ILASP: The complexity and generality of learning answer set programs: https://doi.org/10.1016/j.artint.2018.03.005

How operational examples can be used to learn logical specifications for AI tools.

- Thomson Reuters Imperial Frontier AI Research Lab

- Cambridge Spark: https://www.cambridgespark.com


## Connect With the Show

- Jeremy Bradley on LinkedIn: https://www.linkedin.com/in/jeremy-bradley/

- Cambridge Spark on LinkedIn: https://www.linkedin.com/school/cambridge-spark/


Subscribe to Inside the Algorithm so you never miss an episode. Visit cambridgespark.com to find out how Cambridge Spark can upskill your workforce in data and AI.


#InsideTheAlgorithm #EnterpriseAI #NeuroSymbolicAI #MachineLearning #AISafety

Altri episodi di "Data & AI Mastery"