AirLLM Brings Frontier AI Models to 4GB GPUs, Running 2.8 Trillion Parameter Kimi K3 and DeepSeek-V3

Large open-source AI models can now run on consumer-grade graphics cards through a new inference technique that streams model layers from disk instead of loading them entirely into video memory. AirLLM, an open-source Python library, loads transformer layers one at a time during inference, maintaining only a single layer in GPU VRAM at once. The approach reduces peak GPU memory requirements by over 95%, enabling models that traditionally demand 140GB or more of VRAM to function on hardware with as little as 4GB of memory—without quantization, distillation, or pruning.