Breaking

NVIDIA Releases Nemotron 3.5 Bolt: 74-Billion-Parameter Agent Model Optimized for 24GB Local Deployment

October 16, 2026 at 04:07 EDT

Foundation Models
AI Agents
Open Source

NVIDIA has released Nemotron 3.5 Bolt 74B-A9B, a 74-billion-parameter mixture-of-experts language model designed to run advanced AI agent tasks locally on consumer and enterprise hardware with just 24GB of memory. The model became available on October 15, 2026, across major platforms including Hugging Face, with broad framework support for vLLM, Ollama, llama.cpp, LM Studio, and Unsloth.

October 15, 2026 · NVIDIA

74B Agent Model Fits in 24GB

NVIDIA releases Nemotron 3.5 Bolt, a locally-deployable large language model ranked #1 for agentic reasoning among open-weight models, using sparse mixture-of-experts and 2-bit quantization.

74B

Total Parameters

Active Per Token

24GB

Memory Footprint

80+ TPS

Throughput

What Makes It Possible

Mixture of Experts: Only 9B of 74B parameters activate per input, reducing computation by ~88%. Quantization: Native 2-bit precision shrinks weights while preserving reasoning quality. Compression: Derived from the 120B Nemotron 3.5 Super VL via architecture search.

Practical Context

24GB is the model weights figure. Real inference adds KV cache memory for longer conversations, runtime overhead, and agentic frameworks—typically 32GB+ total. Developers recommend a hybrid approach: Bolt for routine tasks, cloud APIs for the hardest 10%.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.