Breaking

Portable-AI-USB Runs Local AI Offline From a Flash Drive, Released on GitHub

July 7, 2026 at 20:46 EDT

Open Source
Foundation Models

Portable-AI-USB, a project that combines Ollama and AnythingLLM to run local AI fully offline from a USB flash drive, has been released on GitHub. Built with a "zero-trace" design that leaves no settings or data on the host PC, it supports Windows, macOS and Linux.

Open Source · Portable-AI-USB

A Full Local AI That Runs Entirely From a USB Stick

Ollama + AnythingLLM packed onto an exFAT flash drive: plug into any Windows, Mac or Linux PC and chat offline — no install, no registry entries, no file traces left behind.

2.1k

GitHub stars

537

forks

100%

offline after first run

model choices (GGUF)

USB capacity you need

Taller column = more storage. All six models total about 25GB.

16GB

one 3B / 3.8B
lightweight model

32GB

NemoMix 12B
alone

64GB

all six models
(~25GB)

Bundled models · GGUF for CPU inference

NemoMix Unleashed 12B UNCENSORED · rec. ~7.0GB

Dolphin 2.9 Llama 3 8B UNCENSORED ~4.9GB

Qwen 2.5 7B Instruct ~4.7GB

Mistral 7B Instruct v0.3 ~4.1GB

Phi-3.5 Mini 3.8B light ~2.2GB

Llama 3.2 3B Instruct light ~2.0GB

Custom GGUF — direct Hugging Face links —

How it runs

Copy to USB

exFAT drive, run install.bat / install.sh

→

First run online

Downloads models + AnythingLLM

→

Plug in anywhere

Fully offline, no traces on host

CPU inference · ~10–30s responses · 4K context (adjustable to 8K) · 8GB RAM for 12B, 6GB for 7B

What developers praise

Plug-and-play convenience, zero install
Choice of uncensored models
Multi-model + AnythingLLM RAG self-contained on the drive

Real-world friction

CPU-only speed — best kept to lightweight models
Manual in-USB folder step at first install
JavaScript errors when mo

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.