Conspicuous Cognition Podcast podcast

Should We Care About AI Welfare? (with Robert Long)

0:00
1:28:36
Rewind 15 seconds
Fast Forward 15 seconds

Almost all of the discussion about the risks associated with AI focuses on the dangers that increasingly advanced AI systems pose to us — to humanity. But what about the dangers that we might pose to them? As these systems become increasingly intelligent and agentic, AI companies, policy makers, and ordinary citizens need to start taking the possibility of AI consciousness and welfare seriously. If we are in the process of bringing complex and sophisticated minds into existence, how should we understand and treat such minds?

In this episode, Henry and I discuss these issues with Robert Long, founder and executive director of Eleos AI, a research nonprofit dedicated to understanding and addressing the potential wellbeing and “moral patienthood” of AI systems. Rob did his PhD in philosophy at NYU under David Chalmers, and is the co-author of two of the most important papers in the emerging field of AI welfare: “Consciousness in Artificial Intelligence” and “Taking AI Welfare Seriously”.

This was a really fun, informative, and wide-ranging conversation. Among other topics, we discussed:

* Why Rob disagrees with previous guest Anil Seth in taking the possibility of AI consciousness very seriously.

* Why “fancy autocomplete” dismissals of large language models miss the point, and what, if anything, we can learn about an AI model’s experiences by talking to it.

* The difference between consciousness and the kinds of motivations and interests that might actually ground moral status, and whether AI systems could have one without the other.

* What Rob found when he conducted the first externally-commissioned welfare evaluation of a frontier AI model, Claude, and why Claude appears to have an inflated self-conception of what it wants.

* Rob’s experiments with Claude Mythos, an AI model so advanced it hasn’t been released to the public yet.

* Why the fact that Anthropic writes Claude’s character arguably doesn’t settle whether Claude has genuine preferences and values — and the difficult philosophical questions this throws up.

* The “willing servitude” problem: if we succeed in building AI systems that genuinely love being helpful, is that a good outcome or a horrifying one?

* How AI welfare connects to AI safety, and why caring about model wellbeing may turn out to be pragmatically important for alignment even if you’re skeptical about AI consciousness.

* Why AI welfare is already becoming a political and legal battleground.

* Practical advice for users: whether it’s worth being polite to your chatbot, and what low-cost things you can do if you want to hedge against the possibility that these systems might matter morally.

* Whether discourse about AI consciousness functions as hype or propaganda for AI companies, and why Rob thinks AI companies actually have an incentive to downplay AI consciousness.

Links and further reading

* Eleos AI Research — Rob’s nonprofit. Home to their research agenda, team page, and blog. If you want to follow the institutional effort on AI welfare, start here. They’re also, as Rob mentioned in the episode, actively fundraising and hiring.

* “Taking AI Welfare Seriously” (Long, Sebo, Butlin et al., 2024) — the flagship report, co-authored with Jeff Sebo, David Chalmers, Jonathan Birch, and others. Argues that there’s a realistic near-future possibility of conscious or robustly agentic AI systems, and lays out concrete steps AI companies should be taking now.

* “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” (Butlin, Long et al., 2023) — the “indicators” paper referenced several times in the episode. Surveys leading neuroscientific theories of consciousness and derives computational properties you’d look for in an AI system. S

* Rob’s Substack, Experience Machines — where Rob writes more informally. The piece we discussed in the episode, “Language models are different from humans, and that’s okay,” is a good entry point, as is his “Can AI systems introspect?”.

* Anthropic’s “Exploring model welfare” post — the research program under which the welfare evaluations Rob discusses were conducted. Relevant both as a primary source and as evidence that at least one major lab is treating these questions as more than an academic curiosity.

* Henry’s “Consciousness, Machines, and Moral Status” — Henry’s paper arguing that debates about AI consciousness are unlikely to be settled by the science of consciousness alone, and will instead be shaped by shifts in public attitudes as social AI becomes more widespread. Closely related to the public-opinion thread toward the end of the episode.

* Henry’s “All too human? Identifying and mitigating ethical risks of Social AI” — Henry’s broader survey of the ethical terrain around conversational AI systems designed for companionship, romance, and entertainment. Useful background for anyone who thinks the “AI girlfriend” phenomenon is a fringe concern.

* Rob’s long conversation with Luisa Rodriguez on the 80,000 Hours podcast — a three-and-a-half-hour deep dive if you want to hear more from Rob.

Transcript

(Please note that this transcript was lightly AI-edited and may contain minor mistakes)

Henry Shevlin: Welcome back. I’m thrilled to say that our guest today here on Conspicuous Cognition is Robert Long — or Rob, as he’s known to friends — one of the most important people thinking about AI and moral status on the planet right now. Rob is the founder of Eleos AI, a research nonprofit that, in the space of about 18 months, has dragged the question of whether AI systems might one day be moral patients from the philosophical wilderness into the boardrooms of frontier AI labs.

He’s the co-author of “Taking AI Welfare Seriously,” as well as the landmark “Consciousness Indicators” paper with Patrick Butlin and other authors. Rob also conducted the first ever officially commissioned welfare evaluation of a frontier model. Before Eleos, he was at the Center for AI Safety and at the Future of Humanity Institute, and he did his PhD at NYU with Dave Chalmers. He’s also, I should say, one of my favourite interlocutors on these questions anywhere in the world, and I’ve been looking forward to this conversation for months. So Rob, welcome.

Robert Long: Thanks so much, Henry. Likewise — and Dan, it’s great to meet you. I’ve been following your work. I’m really excited to talk to you about these issues.

Henry: Fantastic. So for people who aren’t familiar with Eleos AI, can you tell us a little bit about what it is and how it came about?

Rob: Yeah, so I guess we have been around for 18 months. When you said that number, I was like, whoa, has it really been that long? Time is just so weird when you work on AI. That was, I don’t know, a billion years in AI progress time, but also it feels like it was just last week in my personal life.

Anyway — Eleos Research is a research nonprofit. We’re about four people. We work on the question of when and whether AI systems will be conscious or otherwise merit moral consideration, with a special focus on what we should do now: collectively, as a society, as AI companies, as policymakers. We think this is an extremely neglected issue. We’re building these really complicated AI systems. They kind of look like minds, but we don’t really understand their potential welfare. So we’re just trying to make progress on this and get more people to take it seriously.

It got started because I was beginning to work on these issues organically — I’d worked on them as a philosopher, I’d worked on them at the Future of Humanity Institute. But Anthropic had actually approached me and some colleagues for advice on these issues. And in the first instance, I was having logistical problems hiring a team and assembling a team as an individual. Someone suggested I have my own bank account, or some way to pay people. And then Eleos kind of organically grew out of that and has now grown into a fully-fledged org in its own right.

Henry: Out of interest, Rob — is there any degree to which this was motivated or informed by your personal in…



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

More episodes from "Conspicuous Cognition Podcast"