AI Bites: The Academic Series podcast

Title: EP 61 | MIT 6.036: Gradient Descent & Optimization in Practice

0:00
32:39
Rewind 15 seconds
Fast Forward 15 seconds
  • Welcome to Week 5 of MIT 6.036! Now that we've framed machine learning as loss minimization, how do we actually find the parameters that sit at the bottom of the bowl? We explore the geometry of loss landscapes, analyze step-size dynamics, derive analytical gradients for Logistic Regression, and examine why Stochastic Gradient Descent (SGD) uses random noise to outpace traditional Batch GD.

  • Key Topics:

    • Intuition and update rules for 1D and High-Dimensional Gradient Descent.

    • Convexity, Local Optima, and the critical role of learning rates ($\eta$).

    • Computing analytical gradients for Logistic Regression ($\nabla_\theta J_{lr}$ and $\frac{\partial J}{\partial \theta_0}$).

    • Stochastic Gradient Descent (SGD) convergence conditions and why noise prevents local traps.

  • Disclaimer: Note: This is an AI-generated discussion created using Google's NotebookLM, Gemini, and other AI tools, based on the freely and publicly available MIT 6.036 - Introduction to Machine Learning course material and personal study notes.

More episodes from "AI Bites: The Academic Series"