TL;DR - A pair of open-source engines, Colibrì and DwarfStar, now run genuinely frontier-size AI models, including a 744-billion-parameter one, on hardware you already own, by treating your SSD as an extension of RAM. The trick is that modern open-weight models only "wake up" 5-7% of their parameters per word, so you never need the whole thing loaded at once. It is slow on modest gear and gets fast on decent gear, but the real story isn't speed. It's that your sensitive data can now go through a frontier-class model without ever leaving your building. What you need to do: understand why this works, know its honest limits, and start thinking about which of your data belongs on a model you own rather than one you rent.
By The Numbers
| Thing | Figure |
|---|---|
| Parameters in GLM-5.2 (the model Colibrì runs) | 744 billion |
| RAM Colibrì needs to run it | ~25 GB |
| Share of GLM-5.2 kept resident in RAM (the "dense core") | ~10 GB |
| Size of the "experts" streamed from SSD on demand | ~370 GB |
| Share of total parameters actually used per token | ~5-7% |
| Colibrì's GitHub stars | ~8,000 |
| DwarfStar (ds4) generation speed on a MacBook Pro M5 Max | ~35-40 tokens/sec |
| Cost per token on a model you own | $0 |
The idea that makes this possible
I've spent fifteen years telling people that real AI power meant sending your data to someone else's server. There simply wasn't another option, a model good enough to be worth using was too big to fit on anything you owned. That assumption just broke. A 744-billion-parameter model, the kind of size that used to mean "data centre only," now runs on an ordinary PC with about 25GB of RAM, and nothing about it needs to touch the internet once it's downloaded.
What changed is not a hardware breakthrough. It's a shape of model called Mixture of Experts, MoE for short, and most of the serious open-weight models released this year use it. Instead of one enormous block of parameters that all fire for every single word, an MoE model is built from many smaller "expert" sub-networks, and a router picks a handful of them for each token it generates. For GLM-5.2, the model in question, that means only around 5 to 7 percent of its 744 billion parameters are actually doing work at any given moment.
Once you know that, the old assumption falls apart. You don't need the whole 744-billion-parameter model sitting in memory. You need the small part that's currently in use, plus a fast way to fetch the next part the moment the router asks for it, and two separate teams have now built engines that do exactly that.
Colibrì: a 744-billion-parameter model on 25GB of RAM
Colibrì is an open-source inference engine written by a developer who goes by JustVugg, released as pure C under an Apache 2.0 licence and already sitting around 8,000 stars on GitHub. It runs GLM-5.2, the 744-billion-parameter model, on a normal consumer machine with about 25GB of RAM. Gigazine covered the release when it hit Hacker News.
Here's how. Colibrì keeps the "dense core" of the model, the roughly 10GB of parameters that fire on every token regardless of what's being asked, resident in RAM the whole time. The other ~370GB, the mixture-of-experts layers, live on the SSD and get streamed in on demand as the router selects them. Your storage drive becomes, in effect, a very large and reasonably fast extension of your RAM.
I want to be honest about what that costs you: speed. On a cold cache, or on modest hardware, output crawls. We're talking fractions of a token per second, the kind of pace where you start a request and go make breakfast. On a fast NVMe drive with a warmed-up cache, it becomes genuinely usable. Slow is not the same as broken, and it keeps getting faster with every release.
DwarfStar: antirez brings the same idea to DeepSeek V4
The second engine is DwarfStar, also called ds4, built from scratch by Salvatore Sanfilippo. If that name doesn't ring a bell, he's the guy who created Redis, one of the most widely deployed pieces of open-source infrastructure on the internet. When someone with that track record builds a local inference engine, I pay attention.
DwarfStar runs DeepSeek V4 locally and takes the SSD-streaming idea a step further: it treats the SSD-stored KV cache (the running memory of a conversation, not just the model weights) as what antirez calls "a first-class disk citizen," rather than a bolt-on afterthought. His framing of what this actually changes is the best explanation I've read of why this matters:
SSD streaming turns available RAM "from a hard cutoff (can I run this model or not?) to a continuous spectrum of speed levels."
It used to be pass/fail. Now it's a dial. On a MacBook Pro M5 Max, DwarfStar reportedly generates DeepSeek V4 Flash output at around 35 to 40 tokens per second, which is genuinely usable for real work, not a demo you screenshot and never touch again.
Why this is a privacy story, not just a tech story
This is the part I actually care about. For the last few years, using a genuinely capable AI model has meant one thing in practice: sending your data to someone else's cloud. You type your question, your document, your customer list, your medical history, into a box, and it travels to a server you don't control, run by a company whose terms of service you probably didn't read closely.
That was the trade-off, and for most people it felt like the only option, because a model good enough to be useful was too big to run at home. Colibrì and DwarfStar are proof that trade-off is no longer absolute. A frontier-size model can now run entirely on hardware you own, streaming its own weights off your own SSD, with nothing going over the network at all.
For a small business holding customer records, or anyone working with health, legal, or financial documents, that's not a nerdy footnote. That's the data-sovereignty control the cloud quietly took away from us, handed back. You can now put a genuinely capable model to work on your most sensitive material without it ever leaving the building.
What this doesn't fix
I don't want to oversell this, because the hype around AI is exhausting enough without me adding to it.
Running a model locally solves the network-privacy problem. Your data doesn't leave your machine to reach the model. It does not, on its own, make AI safe to use carelessly. A model running on your computer still runs with your access and your permissions, the same way I wrote about when I looked at why your AI agent runs as you. If you wire a local model into a tool that can read your files or run commands, the fact that it never phones home doesn't stop it from making a mess of your own machine. You still need to think about what it can touch.
You also still shouldn't blindly trust what it tells you. Local doesn't mean more accurate. It means private. Those are different problems, and only one of them just got solved.
And honestly, it's slow enough today, even on good hardware, that it complements a cloud model rather than replacing it for most people. I'd run the sensitive, private work locally and keep the big rented frontier model for the tasks that genuinely need its full weight and speed. That's not a compromise, it's just matching the tool to the job.
Key Takeaways
- Mixture of Experts is the enabling idea. Only 5-7% of a model's parameters fire per token, so you never need the whole thing in RAM at once.
- Colibrì runs a 744-billion-parameter model on ~25GB of RAM by keeping a ~10GB dense core resident and streaming ~370GB of experts off the SSD.
- DwarfStar (ds4), built by Redis creator antirez, does the same for DeepSeek V4, and turns RAM from a pass/fail cutoff into a speed dial.
- The real win is data sovereignty. A frontier-size model can now run entirely on hardware you own, with nothing crossing the network.
- Local isn't automatically safe. It still runs with your permissions, and you still need to verify its output. It's a privacy fix, not a trust fix.
FAQ
Q: Do I need a powerful gaming PC to try this?
No. Colibrì is built to run on an ordinary consumer machine with about 25GB of RAM and a decent SSD, no graphics card required. It will be slow on modest hardware, but it runs.
Q: Is this actually as good as using ChatGPT or Claude in the cloud?
For the hardest reasoning tasks, cloud frontier models still have the edge, and they're faster today. What's changed is that a genuinely capable, frontier-size model is now an option you can run entirely yourself, for the work where privacy matters more than raw speed.
Q: What's actually different about streaming from an SSD versus just not having enough RAM?
Without this approach, a model that doesn't fit in RAM simply fails to load, or the operating system thrashes trying to swap it in and out badly. Colibrì and DwarfStar are purpose-built to fetch only the small slice of the model each token actually needs, in the order it needs it, which is a fundamentally different (and much faster) access pattern than generic memory swapping.
Q: Should my small business switch everything to a local model?
Not everything, and not yet. Keep the routine, low-stakes work on whatever cloud tool you already use. Move the genuinely sensitive material, customer records, health data, anything with a privacy obligation attached, onto a local setup where the data never leaves your premises.
My Take
The cloud sold itself as the only way to get real computing power, and for a long time it was right. Somewhere in that deal, most of us stopped asking where our data actually goes once we hit send, because there wasn't a practical alternative anyway.
Colibrì and DwarfStar are two small, scrappy, open-source projects, one from a developer most of us have never heard of, one from the person who built Redis, and between them they've quietly proven that the "hard cutoff" excuse doesn't hold anymore. Slow today, sure. Getting faster every month, also sure. But the fact that it works at all on gear you already own is the headline, not the tokens-per-second number.
I'm not throwing out my cloud AI subscriptions. I am starting to think seriously about which files on my machine belong on a model that never has to ask permission to leave the building.
Stay safe out there, Mat C
Mathew Clark Founder, SecureInSeconds Currently: working out which files never need to leave the building.
Further Reading
- Your AI agent runs as you: harnesses, open weights, and the security question under the hype - what "local" doesn't fix by itself.
- Harness over frontier: running production AI agents on open-weights models - the cost and lock-in side of the same open-weights shift.
- Zero trust for normal people - the same "who can actually reach this" thinking, applied to identity and access.
- The Microsoft Copilot security disaster nobody saw coming - what happens when vendor convenience quietly becomes your architecture.
- Free security tools that actually work - more open-source options for people who want to own their own stack.
