Marginalia podcast

PlaNet: Learning Latent Dynamics for Planning from Pixels (Danijar Hafner et al.)

16.11.2026
0:00
55:32
Spol 15 sekunder tilbage
Spol 15 sekunder frem

Danijar Hafner and colleagues introduce PlaNet, an agent that learns dynamics from images and plans in latent space.

Its recurrent state-space model combines deterministic memory with stochastic states.

Planning compares predicted action sequences before acting and updating from new observations.

The paper studies latent overshooting and evaluates control across six simulated tasks.

The discussion examines sample efficiency, model uncertainty, and the limits of planning with learned predictions.

Technical clarification: CEM samples a Gaussian distribution, which is not uniform over actions. The variational objective bounds log evidence and includes reconstruction, reward prediction, and KL terms; KL divergence compares distributions rather than literally stretching bell curves. The six-task experiment establishes neither zero-shot transfer nor inevitable failure on every unseen task.

Author: Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee and James Davidson

Source: Learning Latent Dynamics for Planning from Pixels · Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee and James Davidson

Series: Weekly AI Papers · Historical classic

Explore: AI & Computer Science · AI Papers

Related episodes: Unsupervised Learning for Physical Interaction through Video Prediction (Chelsea Finn, Ian Goodfellow & Sergey Levine) · La-JEPA: Compact Robot Policies with Latent World Modeling (Mateo Clémente et al.) · World Models and V-JEPA: Learning from Video (Yann LeCun & Adrien Bardes et al.)

Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode!

Flere episoder fra "Marginalia"