Local AI Chat — an LLM Running Entirely in Your Browser
Run a real large language model on your own GPU, inside your browser. Pick a model, let the weights download once (they’re cached), and chat with streaming responses — completely offline-capable after the first load. No API key, no account, and not a single token of your conversation leaves your device.
Use Cases
How to run an AI chatbot locally in your browser
The tool needs WebGPU — built into Chrome and Edge 113+ and recent Safari. If your browser lacks it, you’ll see a clear message with alternatives.
Choose Qwen2.5 0.5B (fastest), Llama 3.2 1B (balanced), or Phi-3.5 Mini (most capable). Click Load — weights download once with a progress bar and are cached by your browser.
Type a message and watch the reply stream in, rendered as markdown, with a live tokens-per-second readout. Use Stop to cut a response short.
Edit the system prompt to change the assistant’s behavior, and export the whole conversation as JSON at any time.
Frequently Asked Questions
- Does my conversation get sent to a server?
- No. The model runs on your GPU via WebGPU. After the one-time weight download, inference is fully local — prompts and responses never leave your device. You can even go offline and keep chatting.
- What is WebGPU and which browsers have it?
- WebGPU is the browser API that gives web pages access to your GPU. Chrome and Edge 113+ ship it enabled, recent Safari supports it on Apple Silicon, and Firefox is rolling it out. Without it, the model cannot run.
- How big are the model downloads?
- Roughly 350MB for Qwen2.5 0.5B, 600MB for Llama 3.2 1B, and 2.3GB for Phi-3.5 Mini. Weights are cached by the browser, so you only download each model once.
- Which model should I pick?
- Start with Qwen2.5 0.5B to verify everything works — it loads fastest. Llama 3.2 1B is the best speed/quality balance for most machines. Phi-3.5 Mini gives the strongest answers but wants ~4GB of GPU memory.
- Why is it slower than ChatGPT?
- You’re running the model on your own hardware instead of a datacenter. On a recent laptop GPU expect roughly 10–40 tokens/s from the small models — the tokens-per-second readout shows your actual speed.
- Is this as capable as GPT-4 or Claude?
- No — these are 0.5B–3.8B parameter models, great for drafting, summarizing, quick questions, and private experimentation, but well below frontier cloud models. The trade is absolute privacy and zero cost.
The three models, honestly compared
All three models run 4-bit quantized through the MLC/web-llm runtime on WebGPU. The right choice is a function of your GPU memory and patience: smaller downloads and faster tokens versus stronger reasoning. Numbers below are typical for a modern laptop; a discrete GPU roughly doubles throughput.
| Model | Download | GPU memory needed | Best at |
|---|---|---|---|
| Qwen2.5 0.5B Instruct | ~350MB | ~1GB | Instant-feel replies, quick Q&A, low-end hardware and iGPUs |
| Llama 3.2 1B Instruct | ~600MB | ~1GB | Everyday drafting and summarizing — the balanced default |
| Phi-3.5 Mini 3.8B | ~2.3GB | ~4GB | Multi-step reasoning and code questions — needs a discrete GPU or Apple Silicon |
- Weights are fetched from HuggingFace by the MLC runtime on first load and cached via the browser Cache API — subsequent loads skip the download entirely. Clearing site data evicts the cache.
- The model list is resolved at runtime against the web-llm library’s own prebuilt catalog, so the exact build ids track library upgrades instead of going stale.
Private by design. The AI model runs on your own GPU via WebGPU — prompts and responses never leave your device. The only network traffic is the one-time model weight download (from HuggingFace), which is then cached by your browser.