
NVIDIA just shipped a model with four times the context window of what's been running my home agent for months. So I swapped it in, broke my own setup within the first hour, and then threw 700,000 words at it to see if it would lie to me.
No guest this episode, just me and my home lab. Jarvis, my home agent, has been running on Qwen3.6-35B-A3B since the start, served through vLLM on the DGX Spark's 128GB of unified memory. Then NVIDIA released Nemotron 3.5 Lightning 30B-A3B in NVFP4, and one number stopped me mid-scroll: a validated 1 million token context window, four times what Qwen gives you at 262,000.I don't trust a spec sheet claim until I've broken it myself, so I swapped the model into Jarvis, found a memory reservation bug in NVIDIA's own deployment recipe within the first hour, fixed it, and then buried a fabricated secret code word 90% of the way through a 700,000 word document to see if the model could actually recall it. I also dug into DSpark, the speculative decoding checkpoint built specifically for this hardware, and measured what it actually does for throughput.
This episode is the honest version: what's proven on my own hardware, what's still sitting on NVIDIA's spec sheet, and what I still don't know about whether Nemotron makes Jarvis a better agent day to day.
Key topics covered:
- Why Jarvis's setup (Qwen3.6-35B-A3B on vLLM) needed an indirection layer before any of this mattered
- NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4's validated 1 million token context window, tested against a 700,000-word document
- The memory reservation bug in NVIDIA's DGX Spark deployment recipe, and the one-line fix
- DSpark speculative decoding: how it works, and the acceptance rate that made it worth using
- Nemotron's hybrid Mamba-2, attention, and mixture-of-experts architecture, and why it matters for running 1M tokens on a single box
- What's proven on real hardware versus what's still just NVIDIA's own benchmark numbers
- Why Nemotron isn't confirmed as the better agent brain for Jarvis yet
Chapters
00:00 Introduction — what a 1M-token context window actually changes
00:37 Welcome back to The Private AI Lab (no guest this episode)
01:19 Recap: Jarvis running on Qwen3.6-35B-A3B via vLLM on the DGX Spark
01:58 NVIDIA Nemotron 3.5 Lightning lands with a validated 1M-token window
02:49 Deployment: why the Jarvis-LLM alias made the model swap painless
03:28 The memory reservation bug NVIDIA's own recipe doesn't warn about
04:53 The test: a 700,000-word document with one hidden code word
05:51 DSpark speculative decoding: free speed, fully verified
06:47 The reasoning quirk that quietly eats your token budget
07:31 Nemotron vs Qwen: what's proven versus what's still a spec sheet claim
08:01 The hybrid architecture behind why 1M tokens fits on a DGX Spark
09:24 NVIDIA's own SWE-bench Verified numbers, and their limits
09:56 Closing thoughts, and what's coming in the next episode
Links mentioned in this episode:
- NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4 on Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
- NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4-DSpark (speculative decoding checkpoint): https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark
- Qwen3.5-35B-A3B on Hugging Face: https://huggingface.co/Qwen/Qwen3.5-35B-A3B
- NVIDIA DGX Spark: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
- vLLM: https://github.com/vllm-project/vllm
Flere episoder fra "The Private AI Lab"



Gå ikke glip af nogen episoder af “The Private AI Lab” - abonnér på podcasten med gratisapp GetPodcast.








