Breaking

RightNow-AI PicoLM Runs TinyLlama 1.1B on $10 Boards in 45 MB of RAM

September 13, 2026 at 13:11 EDT

Open Source
Foundation Models
AI Agents

RightNow-AI's PicoLM is a from-scratch C11 inference engine that runs TinyLlama 1.1B on development boards priced around $10, using about 45 MB of runtime RAM and a single binary of roughly 80 KB, with no Python runtime and no cloud API.

2026 · RightNow-AI

A 1.1B LLM on a $10 board

PicoLM, a ~2,500-line C11 engine, runs TinyLlama 1.1B from a 638 MB disk file using about 45 MB of RAM and an ~80 KB binary — no Python, no cloud.

Board price

$10

LicheeRV Nano

Runtime RAM

45 MB

2,048-token KV cache

Binary

~80 KB

Zero extra libraries

Model

1.1B

TinyLlama Q4_K_M

Generation speed, TinyLlama 1.1B

Tokens per second. Column height is 1 block per tok/s. Pi 4 and Pi Zero figures are NEON estimates.

LicheeRV
$10

Pi Zero
$15

Pi 3B+
$25

Pi 4
$35

~10

Pi 5
$60

x86 path: 1.6 tok/s to 13.5 tok/s

About 8× after fused dots, threads, and SIMD. Eight stacked blocks vs one.

1.6

Naive scalar

13.5

8 threads, x86

What works

Offline on 256 MB boards. ~80 KB binary. JSON grammar for tool calls. 74% faster reruns when the KV cache is saved.

Limits

TinyLlama-level answers, CPU only. One LicheeRV Nano run took 16 min 20 s for 10 tokens, far below ~1 tok/s.

How the 638 MB file fits

Weights stay on disk; RAM holds the working set

mmap pages in about one layer at a time. Default 2,048-token FP16 cache still uses ~44 MB — tight if Linux only sees 128 MB of a 256 MB board.

638 MB GGUF on disk→mmap one layer→45 MB runtime→next token

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.