Innhold Show
Edge0-145B makes it possible to run a 145-billion-parameter AI agent locally on one powerful PC. On an RTX 4090, the developers report 69.9 tokens per second in chat and 91.7 tokens per second in agent mode. On paper, this is a local AI milestone, and those numbers make me sit up and pay attention.
The trick is that the entire model does not have to live in video memory at the same time. Edge0 combines VRAM, system memory and a sparse Mixture-of-Experts architecture in which only a small part of the model works on each token. That allows a model file larger than 40 GiB to run with a graphics card that has 24 GiB of VRAM.
This is exactly the direction I want to see: powerful AI agents that work locally, use tools and keep data on your own machine. But “frontier” is still the developers’ label, the performance figures come from Edge0, and the company’s launch material includes some practical catches. Those catches do not make the model less exciting. They determine who should actually download it today.
What is Edge0-145B?
Edge0-145B is a local agent model with 145 billion parameters, 10.12 billion active parameters per token and a 256,000-token context window. The model is derived from Qwen3.8-Flash-Next and comes in Q3_K and NVFP4 formats, according to the official model card on Hugging Face.
“Active parameters” is the key phrase. This is a sparse Mixture-of-Experts model with 512 experts in each layer. For every token, the router selects 10 experts while one shared expert is also active. You download an enormous model, but the machine does not process all 145 billion parameters for every word.
The model is designed for more than questions and answers. Edge0 demonstrates workflows in which it plans, writes and runs code, detects errors and evaluates the result visually. The examples include Blender scenes, games, video production and circuit-board design. That puts it in the same exciting local-agent category as Meta’s frontier model Muse Wonder, but with a very different hardware story.
How does Edge0-145B run with 24 GiB of VRAM?
Edge0-145B fits by sharing the workload between GPU memory and ordinary system memory. The Q3_K version contains 40.81 GiB of model weights. In Edge0’s test with residency enabled, it used 23.14 GiB of VRAM and 31.28 GiB of RAM. Those numbers cannot be added as though they represented two copies of the same data; different sets of experts live in the two memory tiers.
The architecture compresses the experts’ input to a latent width of 1,792 while the attention component retains its full width. The 512 experts are small, fixed blocks that the engine can move between system memory and the graphics card as needed. A separate 1.26-billion-parameter MTP model predicts multiple tokens ahead, allowing speculative decoding to get more work from each load.
That whole system is what matters. “145B on one GPU” can sound as though the entire model has somehow been squeezed into 24 GiB of VRAM. That is not what is happening. The machine uses both the graphics card and system memory, while the engine makes the right experts available when they are needed. Smart software and a model designed for local inference are doing the job together.
Edge0 also tested a constrained setup with 16 GiB of VRAM and 20 GiB of RAM. Chat speed fell to 36.8 tokens per second and agent speed to 44.9. That shows 24 GiB is not an absolute lower limit in the company’s test setup, but it does not guarantee that every 16 GiB GPU will work. The measurement came from an RTX 5090 D restricted with cgroups.
How fast is Edge0-145B on an RTX 4090?
On an RTX 4090 with 24 GiB of VRAM, Edge0 reports prefill speeds of 2,042 tokens per second with 4,000 input tokens and 3,179.4 with 16,000 tokens. It reports 69.9 tokens per second in chat and 91.7 in agent mode. The project’s GitHub documentation also lists 99.7 tokens per second in chat and 121.2 in agent mode on an RTX 5090 D.
Prefill is the speed at which the model reads the input before it starts answering. Decode speed is what you notice while the response appears. Nearly 70 tokens per second in chat on a 4090 is therefore much more than a technical curiosity. If the result holds up in ordinary work, local inference is fast enough to avoid feeling like you are waiting for a giant server model that has wandered off into the basement.
But these are vendor results from a specific build. Storage speed, RAM capacity, memory bandwidth, operating system, prompt length and the number of experts kept in VRAM can all change performance. I have not tested Edge0-145B myself. I am treating these figures as Edge0’s measurements, not as my own confirmation.
For Apple hardware, the model card reports 60.9 tokens per second in chat on an M2 Ultra. The updated documentation also says this is a Metal measurement, not proof that the 145B release is currently available for macOS. The published build instructions for the present release cover NVIDIA hardware with CUDA 13 on Linux or Windows. A benchmark result and a supported release are not the same thing.
Is it really competitive with the best cloud models?
Edge0-145B looks strong in Edge0’s own evaluations, particularly for coding and agent work. The most striking figure is 86.5 on Terminal-Bench 2.1, where the model ranks ahead of both Qwen3.8-Flash-Next and Claude Opus 4.6 Max in Edge0’s table.
That is promising, but one benchmark table is not a license to declare the model the winner. Edge0 says the results for Edge0, Qwen, DeepSeek and Claude come from its internal evaluations. Edge0-145B does not win everything either: Qwen3.8-Flash-Next leads on GPQA-Diamond and IFBench, while Claude leads on Humanity’s Last Exam.
The more important claim is that Edge0-145B combines high capability with a memory design that makes local use realistic. The comparison with Qwen and DeepSeek, however, uses estimated memory figures for standard 4-bit inference. Measured and calculated values are mixed in the same comparison. That belongs in the fine print – preferably fine print that is still large enough to read.
I am excited about what the architecture could mean, not because a single benchmark table has crowned a champion. The real test is whether the model can finish long coding and agent tasks on an ordinary high-end workstation without quality collapsing or the memory system becoming a bottleneck.
What do you need to run Edge0-145B?
The main documented route currently requires Linux or Windows, an NVIDIA GPU and CUDA 13. The model directory runs through Edge0’s own edge0-strata engine using edge0-infer. The Q3_K release is 40.81 GiB, so you also need enough fast storage and system memory for the weights and the experts that do not reside in VRAM.
After building the engine and edge0-infer, the model starts with edge0-infer serve or edge0-infer chat. Server mode exposes both OpenAI-compatible and Anthropic-compatible endpoints locally. That makes the model much easier to connect to existing tools than a system that forces every project to learn an exotic new API.
Windows users can also use Edge0 Chat and select the model directory through a graphical interface. The documentation is fresh, however, and the platform support has already been clarified since launch. I would still treat this as a release for technical users, not as a painless one-click model for everyone with a gaming PC.
If you have the right machine, test it on tasks where you know the correct result: a real codebase, a defect the model has to find or an agent job requiring several tool calls. Measure time to a completed result, how often you have to intervene, RAM and VRAM use, and whether the model stays on course during long jobs. Tokens per second are nice. Finished work is better.
The open-source stack comes with a licensing catch
The inference framework is open source under Apache 2.0, while the Strata component retains the MIT license. The model weights have different terms. Edge0’s own contributions to the model use the Edge0 Community Model License 1.0, and the Qwen components retain the Qwen Community License 1.0.
The Edge0 license allows personal use, research, modification, non-commercial sharing, internal use and commercial use of the model’s outputs. Selling or commercially distributing the model, operating a paid model service, or offering a standalone commercial coding or office assistant requires a separate written agreement with Edge0. Qwen’s terms apply as well.
I would therefore describe Edge0-145B as a model with downloadable weights, not frictionless open source for every commercial purpose. Being able to download, inspect and run the model locally still has real value. But a company building a product on top of it needs to read both licenses before development runs away with the plan.
Why Edge0-145B is a milestone for local AI
Edge0-145B moves the boundary of what one powerful workstation can do. An RTX 4090 is expensive hardware, but it is still a consumer graphics card that can sit in a single PC. A 145B model running at useful speed there is very different from a local demonstration that needs a stack of data-center accelerators.
It echoes the development around Perplexity’s local agent computer on NVIDIA DGX Spark: more of an agent’s work can move from a closed cloud service to hardware you control. The difference is that Edge0 also targets widely owned consumer cards.
I would not buy new hardware based on launch figures alone. I would not dismiss this as another model with a large number in its name either. If you already have a 4090 or 5090, enough RAM and an appetite for local agents, Edge0-145B is one of the most exciting models to measure on your own work right now.
On paper, this is already a local AI milestone. If independent testing confirms the combination of agent quality, speed and memory use, Edge0 has also made a model in this size class practically relevant to people who own one PC. This is local AI the way I like it – not merely private and offline, but powerful enough to do real work.
Frequently asked questions
Can Edge0-145B run on an RTX 4090?
Yes. Edge0 reports 69.9 tokens per second in chat and 91.7 in agent mode on an RTX 4090 with 24 GiB of VRAM. The model also uses system memory, so the graphics card is not the entire hardware requirement. These are vendor measurements and will vary by setup.
How much RAM does Edge0-145B need?
There is no single official minimum for every machine. In one Q3_K measurement, Edge0 used 23.14 GiB of VRAM and 31.28 GiB of system memory. A constrained test with 16 GiB of VRAM and 20 GiB of RAM also worked, but it used a restricted RTX 5090 D.
Does Edge0-145B work on a Mac?
Edge0 measured the Q3_K version on an Apple M2 Ultra using Metal, but the project says that result does not establish current macOS availability. The published build instructions for the 145B release currently cover NVIDIA hardware with CUDA 13 on Linux or Windows.
Can Edge0-145B be used commercially?
Internal use and commercial use of generated outputs are allowed, but paid model services, commercial model distribution and standalone commercial coding or office assistants require written permission from Edge0. The Qwen license also applies, so a commercial user needs to review both sets of terms.
