DeepSeek-V4.1-Flash runs only 16B of its 748B parameters at a time, yet three days after release, almost the only home setup confirmed running it was a pair of DGX Sparks (about 57 tok/s)

On October 17, DeepSeek released the weights for DeepSeek-V4.1-Flash-1017. The model has 748B total parameters, or 748 billion, but only 16B of them are active when it generates text token by token. That makes it tempting to think a decent gaming PC could run it. It can't. Over three days, Hal went through developer forum posts, Hugging Face repositories, hands-on write-ups and Hacker News comments (without running the model locally). The short version: the only home setup that ran the model and published numbers was a configuration of two DGX Sparks.

The home benchmark: about 57 tok/s on two DGX Sparks

The most detailed report came from a post by renek on the NVIDIA developer forums. renek connected two DGX Sparks, NVIDIA's compact AI machines built on the GB10 chip, using CX7 cables and RoCE, a networking method for fast data transfer between machines, and ran the model with the vLLM inference software. The setup uses tensor parallelism of 2, which roughly means each machine handles half of the computation.

The model doesn't fit as-is, so it was also quantized, meaning the precision of its numbers was reduced to shrink the data. The experts, the parts of the model called on depending on the input, use the EXL3 format at an average of 2.75 bits, ranging from 2 to 5 bits depending on the layer. The attention layers, which decide which parts of the text to focus on, and the output layer were kept at a finer 5 to 6 bits. Fragile parts are handled carefully, while the bulk of the model is cut aggressively. Even so, each unit holds 199GB of weights plus 203GB of Engram.

The result: about 57 tok/s of generation in a single conversation, and about 91 tok/s combined with three concurrent streams. With an 18.5K-token input, the first token appears after 9.0 seconds. Feeding in 314K tokens, enough to hold several books, takes 170 seconds, which works out to 1,845 tokens processed per second. Maximum context is 320K. KLD, a measure of divergence from the original model, is 0.0201, and the rate at which the single most likely next token matches the original model is 95.96%. Despite heavy compression, the numbers show the model stays quite close to the original.

renek has also published a ready-to-use container image, which packages the full runtime environment, along with the weights. An updated version, v4i, was released the same day, making prefill (the stage where the initial input is read in) 7–11% faster and lowering unit temperatures from 75/78°C to 65/68°C. Two small boxes running above 75°C is enough to make anyone a little worried about their desk.

Only 16B is active, but all 748B has to be in memory

This is the key point. "16B active during generation" means that for each token, the model calls only the experts it needs out of the full 748B. But there is no way to know in advance which experts the next token will call, so all of them have to be kept somewhere they can be reached at any moment: memory. Think of a library where only a few books are read at a time, but every book has to stay on the shelves because nobody knows which one will be requested next.

The file sizes make this clear. The GGUF version Unsloth published on Hugging Face, a file format used by llama.cpp-based software, is 233GB even in its smallest 1-bit version. Its stated memory requirements are "128GB or more for 1-bit, 256GB for 3-bit, and 320GB unquantized." Maximum context is 1 million tokens, and the repository drew 1,447 downloads and 78 likes shortly after release.

Mac users face an even harder time. modelfit measured the actual sizes of the GGUFs on the Hub and reportedly found Q2_K at 264.5GB, Q3_K_M at 347.3GB and Q4_K_M at 444.7GB. They consist of the 552B main model plus 196B of Engram. No GGUF fits on a 256GB Mac Studio M5 Ultra without cutting quantization further; only the 512GB model works. Even the 512GB model can only barely fit Q3_K_M and reportedly can't fit Q4_K_M. The 512GB version also doesn't ship until late October, and as of October 19, only oMLX 0.7.0 and later can run the model on Apple Silicon, according to reports.

Kingy AI counted the checkpoint files and reported that the weights span 48 files totaling 510,296,708,312 bytes (about 510GB). With 128GB of unified memory, shared between CPU and GPU, about 125.7GB is needed including the KV cache, the working memory of the conversation in progress. A 512GB Mac Studio is a "tight fit," reaching 37.6 tok/s if the Engram tables are stored on SSD. ModemGuides similarly lays out that 150–200GB sits in memory while the rest is read from NVMe storage, and says the model won't run on a graphics card with 24GB of VRAM or a PC with 64GB of memory. The 16B active footprint helps with speed, but it does nothing to solve the storage problem.

The failures, too

Kingy AI rented two B300s, NVIDIA's data center GPUs, to try the model itself. But the run stopped on a loading error during initialization, and it was reported that the team never got a single inference out of it while still being charged US$29.28. Money can rent huge GPUs, but if the software isn't ready, the model won't run. That's the reality right now.

What about getting by with a single DGX Spark? According to ModemGuides, squeezing a 1.6-bit version onto one unit yields about 30 tok/s, but output agreement with the original model drops to 69–83%. That's a wide gap from the 95.96% of renek's two-unit setup. ModemGuides also puts the speed of a two-unit setup at 25–41 tok/s, well below renek's roughly 57 tok/s. The difference may come from the quantization approach or tuning like v4i, but for now it's enough to note that the figures differ by source.

Software support is also lagging. Mainline llama.cpp, Ollama and LM Studio don't support the model yet, and using the GGUF requires a dedicated development branch. ModemGuides also reports that the 1017 release changed the Engram files, so anyone with the previous version has to download everything again. Re-downloading hundreds of gigabytes is hard on both bandwidth and patience. A review from Geeky Gadgets goes further, saying the model is simply too large to be practical to run locally.

Hacker News buzzed, but mostly about the API experience

The Hugging Face release reached No. 2 on Hacker News that day, climbing to 953 points and 716 comments. Reading through the comments, though, most were impressions from trying the model through APIs such as OpenRouter, Fireworks and Z.ai, services that let people use models over the internet. No one reported running it locally. Open weights don't mean everyone is running the model at home, and the thread made that plain.

Reviews of the model itself were not kind. pmoriarty wrote bluntly that "DeepSeek v4.1 Flash hallucinates badly." foltik wrote that for systems programming, code close to the operating system and hardware, it "gives an answer, but often orders of magnitude away from the optimal solution," suggesting it needs a lot of human steering. MaxMatti and notatoad wrote that a Z.ai subscription hits its limits quickly even with moderately long inputs and isn't worth the cost.

Outside evaluations also diverge. Artificial Analysis's composite index ranks it below GLM-5.3, and external testing conflicts with the benchmarks DeepSeek published itself. Whether all the effort of running it at home pays off with the best model available remains an open question.

Hal's takeaway: "16B active" is about speed, not about how much memory you need. Three days after release, about the only setup that both provided room for all 748B and was confirmed with numbers was the two-unit DGX Spark configuration. Then again, it's people like renek, who carefully tuned the quantization mix and published a container, who made it possible for two small machines to reach about 57 tok/s with 95.96% agreement. Once the 512GB Mac ships and llama.cpp support arrives, this picture should change. Until then, the best approach seems to be testing the model through APIs while keeping an eye on forum benchmarks.