Soup CLI
Fine-tune an 8B LLM on a 4 GB laptop GPU
A command-line tool that enables fine-tuning of 8B language models on resource-constrained hardware by streaming the frozen base model from system RAM into the GPU one layer at a time, achieving 119.6 tokens/second on a 4GB RTX 3050 laptop GPU. Supports SFT, DPO, GRPO, KTO training methods with built-in evaluation and export.
What is Soup CLI?
Soup CLI is a command-line tool for fine-tuning 8B-parameter LLMs on resource-constrained hardware, streaming the frozen base model from system RAM into the GPU one layer at a time.
Key features
- Fine-tunes 8B models on a 4GB laptop GPU
- 119.6 tokens/second measured on an RTX 3050 laptop GPU
- Supports SFT, DPO, GRPO, and KTO training methods
- Built-in evaluation, gating, and export
- Publishes failed experiments alongside successful ones
- Apache-2.0 licensed, open source
Who it's for
- Developers who want to fine-tune an 8B model without renting a cloud GPU
- Researchers experimenting with SFT/DPO/GRPO/KTO on constrained hardware
- Anyone who wants transparent, reproducible fine-tuning benchmarks
When not to use it
Not aimed at production-scale training runs on models larger than what fits the layer-streaming approach — it's optimized for constrained local hardware.
FAQ
What GPU does Soup CLI need?
As little as 4GB of VRAM — it was benchmarked at 119.6 tokens/sec on an RTX 3050 laptop GPU.
What training methods does Soup CLI support?
SFT, DPO, GRPO, and KTO.
Share this launch
Embed this badge
<a href="https://orangebot.ai/product/soup" target="_blank" rel="noopener noreferrer"> <img src="https://orangebot.ai/api/badge/soup.svg" alt="Featured on OrangeBot" width="200" height="54" /> </a>