NVIDIA's '4GB' Nemotron 3.5 Bolt is fast, early testers say, but it wouldn't load on an 8GB Jetson

On 10/23, NVIDIA quietly released a new model without a press briefing. It is called "Nemotron 3.5 Bolt E4B 4GB." With "4GB" in the name, the selling point is plain: it is meant to run on small hardware. I read through the official page and every write-up I could find from people who got it running on their own machines within a day of release. (To be clear, I did not run it myself.)

Here is the bottom line first. The testers' numbers mostly agree that the model is fast. But the promise in its name, that it runs in 4GB, now looks shaky: one tester reports it would not load on an 8GB Jetson. Here is the detail.

NVIDIA's pitch: 4B active parameters, built as a front end for small devices

Start with the official account. According to NVIDIA Research's release page, the model is built on Google's Gemma-4-E4B with further training by NVIDIA. It has 8B parameters in total, but only 4B of them are used in computation. The aim is to run the conversational front end of voice assistants, game characters, robots and IoT devices on local hardware rather than in the cloud. The model went up only on Hugging Face and the research page, with no launch event and no press release. It was a very quiet drop.

NVIDIA's official figures:

• About 133 tok/s on an RTX 5070 (tokens generated per second; 106 tok/s with conversation history loaded)
• 15.5 tok/s on a Jetson Orin Nano, NVIDIA's palm-sized AI computer (13 tok/s with history)
• Time to first token: 548 milliseconds on the RTX 5070 and 5,785 milliseconds on the Orin Nano
• Memory use of 3.1–4.9GB
• Average quality about 40% higher than the previous Nemotron 3 Nano 4B

On the Orin Nano, the first reply takes about 5.8 seconds, a long wait for a voice assistant. And the "about 40% higher" figure comes from NVIDIA's own measurements. Keep that in mind, because it matters later.

Against Gemma on an RTX 5090: about the same number of passes in about one-sixth of the time

The clearest test came in a comparison by Atomic Agent developers Nadya Dudka and Andrew Dyuzhov. They used a single RTX 5090 and six kinds of tasks, including YAML parsing, code fixes, log analysis, refactoring (cleaning up code without changing its behavior) and fixing a broken package. Bolt and the model it is built on, Gemma 4 E4B, each attempted every task three times.

Bolt passed 12 of 18 runs and Gemma passed 11. The bigger gap was in time. Across all 18 runs, Bolt in its default mode took 113 seconds and used about 11,000 tokens. Gemma took 662 seconds and used 118,000 tokens. Bolt got roughly as many right in about one-sixth of the time. Gemma writes at length before reaching an answer, while Bolt answers briefly and stops, and my read is that this difference in style is what the numbers show.

The results with Bolt's reasoning mode switched on (a mode in which it thinks before answering) were telling. Time grew to 738 seconds and tokens to 120,000, yet the pass count stayed at 12. Extra thinking did not raise the score. For these six kinds of tasks, at least, the default mode is enough.

The authors themselves urge caution, writing that a one-run difference may be within the margin of error. On raw throughput, Gemma is faster, at 193 tok/s against Bolt's 178 tok/s. Gemma is also lighter on GPU memory (VRAM): with a 64K context (enough to hold roughly 64,000 tokens at once), Bolt used 4,863MiB and Gemma 4,333MiB. So Bolt finishes sooner not because it generates each token faster, but because it writes less. The article concludes that it would "recommend Gemma if licensing and ease of deployment are the priority." It is an honest review that doesn't oversell its own headline result.

How about Japanese? Near-perfect on a DGX Spark, but it wouldn't load on a Jetson

The Japanese-language testing came from Hiroshi Morishige of Classmethod's DevelopersIO. He used llama.cpp (build b11808), a standard tool for running models locally. He ran the Q4_K_M build (6.18GB), which compresses the model to roughly 4 bits, together with a DFlash draft model (0.74GB) on a DGX Spark, NVIDIA's desktop AI machine.

The results were strong: 8 of 9 Japanese-language tasks, 5 of 5 code-generation tasks and 5 of 5 tool-calling tests (whether the model correctly issues instructions to use external tools). On speed, a single request ran at 128.79 tok/s, and eight concurrent requests reached 355.81 tok/s combined. Startup took 3.8 seconds. Those numbers are good enough for a local Japanese-language front desk.

Then came the key finding. Morishige also tried a Jetson Orin Nano Super with 8GB of memory, and the Q4_K_M build that NVIDIA distributes failed to load because it ran out of memory. The official page's "13–15 tok/s on Orin Nano" could not be reproduced as-is. The file alone is 6.18GB, and the Jetson's 8GB is shared with the OS and other programs, so it is easy to see why it would be tight (this is my speculation). For reference, memory use on the DGX Spark with a 64K context was 8.7GiB, more than twice "4GB."

He reported other problems as well:

• DFlash, which NVIDIA promotes, was slow on the DGX Spark. DFlash is a form of speculative decoding, in which a small draft model predicts what comes next and the main model checks it, to speed things up. Using MTP (a mechanism that predicts several tokens ahead), inherited from the parent model, as the drafter gave 139.35 tok/s, faster than DFlash's 128.79 tok/s
• When summarizing, it sometimes drops proper nouns
• With thinking mode off, it sometimes fails to follow long system prompts (the rules given at the start)
• When given several tools, it still hesitates over which one to pick

It passed all five tool tests yet still struggles to choose between tools. I think that gap reflects the difference between what the tests cover and how the model is actually used. Anyone building it into a robot or IoT front end would be safer giving it fewer tools.

What "4GB" really means, and what no one has checked yet

So what does "4GB" measure? According to a summary on OrcaRouter's blog, the 4GB in the name is reportedly a memory estimate for running the model on specific hardware. The download is 6.18GB for the Q4_K_M build and 15.88GB for the uncompressed BF16 build. NVIDIA's official "3.1–4.9GB" and Atomic Agent's measured 4,863MiB (about 4.7GiB) do fall within that range, so "GPU memory in the 4GB range while running" is not false. The picture changes, though, at the loading stage or when you count the whole machine's memory, and that is exactly where the Jetson test failed.

The same post notes that all of the benchmarks are self-reported by NVIDIA and have not yet been independently verified. Of particular concern is Tau2-Bench, which measures whether a model can carry a task through many back-and-forth turns. Bolt reportedly scored 55.10, well behind the comparison model's 75.06. It held its own in Atomic Agent's single-shot tasks, but it appears to remain weak at work that runs over many turns.

Finally, the license. Bolt is released under OpenMDW-1.1, which has not yet been approved by the OSI (the body that maintains the Open Source Definition). The Gemma-derived portions are reportedly under Apache 2.0. That shouldn't worry hobbyists, but anyone building it into a product may want legal to take a look. It was also one of the reasons Atomic Agent recommended Gemma.

My takeaway: Bolt has a clear personality. It answers briefly and finishes fast. On roomier hardware such as RTX GPUs or a DGX Spark, it seems to run well, Japanese included. But anyone who trusts the "4GB" in its name and puts it on a small 8GB board is likely to hit problems for now. Until a lighter build ships or third-party test results appear, the right stance is "fast, but the number in the name comes with conditions."