NVIDIA Nemotron 3.5 Bolt: a 74B Open Model That Fits in 24 GB for Local AI Agents

Running a capable AI agent at home usually means a lot of memory. On October 15, 2026, NVIDIA released three new Nemotron 3.5 models, and the most interesting one for home users is built to fit on a single consumer GPU.

Three models, three sizes

• Nemotron 3.5 Super VL: an open 120-billion-parameter multimodal model for enterprise agents. NVIDIA claims it is up to 5x faster than the next best open model in its class.
• Nemotron 3.5 Bolt 74B-A9B 24GB: a mixture-of-experts model with 74 billion parameters, about 9 billion active per token, using native 2-bit quantization to fit in 24 GB. Available now.
• Nemotron 3.5 Bolt E4B 4GB: a small model for voice assistants and robots, due October 23.

What 24 GB really means

24 GB is the text-only figure. According to the model card, vision adds about 1.68 GB, and practical llama.cpp setups need 32 GB or more for a 65K-token context, or 48 GB for 256K. It runs in llama.cpp, Ollama, LM Studio, Unsloth and vLLM, on GeForce RTX PCs, DGX Spark and the new RTX Spark laptops.

NVIDIA says Bolt 74B ranks first among open models in its class on PinchBench and second on BrowseComp. These are NVIDIA’s own figures; we haven’t tested them. TechRepublic notes the Bolt models target agents that browse the web, write and debug code locally.

Rocks take

This is the most realistic “local agent” release in weeks: an RTX 4090 or 5090-class card can actually load it, unlike the 800B-parameter giants. For a Home Assistant setup, the small Bolt E4B could be the more interesting one, a 4 GB model for a local voice assistant. We’ll try it when it lands on October 23.