Breaking

MicroLLM Lab Puts Seven Tiny Language Models in the Browser Without API Keys

September 29, 2026 at 23:16 EDT

Foundation Models
Open Source
Software Dev & Coding

MicroLLM Lab, a public WebGPU experiment, lets users load and compare seven Q4-quantized language models inside a browser tab without an account, install or API key. The project reports 115 tokens per second for PetitGPT on an Apple M4 and keeps inference local, with model files cached in browser storage.

September 2026 · MicroLLM Lab

Tiny LLMs, local in the browser

A WebGPU lab lets users load seven Q4 language models in a browser tab, benchmark them on their own hardware, and keep prompts off external inference servers.

models in the current catalog

26M-362M

parameter range

API keys required

2048

token context cap for Llama-style packs

Apple M4 speed snapshot

Vertical blocks are proportional at roughly one segment per 10 tokens per second.

PetitGPT
115 tok/s

SmolLM2 135M
66 tok/s

The faster baseline gives up accuracy; the instruct model is slower but passes more objective checks.

How the local loop works

Load Q4 weights

→

Cache in IndexedDB

→

Decode with WebGPU

→

Compare and benchmark

Where it fits

Fast local trials for classification, extraction, routing, autocomplete and hands-on model calibration.

Where it breaks

Tiny models can hallucinate, miss arithmetic, repeat text and fail instructions, so high-stakes answers need a stronger system.

The real story is distribution

A model demo can become a URL: no cloud inference bill, no signup wall, and no install step. The trade-off is that the browser, GPU and model size now define the ceiling.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.