AI in the Field · Nov 16, 2026
Solo developer trains a miniature DeepSeek V3 from scratch on a single laptop in 2.6 hours
Four of DeepSeek V3's core mechanisms fit into about 200 lines and were trained on a MacBook Air. Another user reports a full run on an AMD consumer GPU. The result is a model for studying the architecture, not a capable AI
Haru Misaki · Field Correspondent

Key points
- Solo developer Vaibhaav Tiwari wrote four of DeepSeek V3's core mechanisms (MLA, DeepSeekMoE, auxiliary-loss-free load balancing and MTP) in about 200 lines of JAX. He trained the model from scratch on a MacBook Air M5 (24GB) for 12,000 steps, which took about 2.6 hours
- The model has 36.2 million parameters. Validation loss fell from 5.02 to 3.51. Expert utilization, which ranged from 0.09% to 18.6% without balancing, evened out to between 5.7% and 6.9%. Another user also completed a full run on an AMD RX 7600 XT, which took about 50 minutes
- The developer writes that it is "not a strong language model." The MoE is a simplified version that computes every expert, and at this scale load balancing makes no visible difference to accuracy. The distributed trained weights use a format that can run code when loaded, so users should be careful with them
A solo developer has trained a miniature version of DeepSeek V3, the large AI model from China's DeepSeek, from scratch on a single MacBook Air. The miniature includes all of the original's core mechanisms. Training took about 2.6 hours, and the code runs to about 200 lines. We went through the public repository, its README and the discussion on Hacker News, the forum popular with engineers. In short, running and testing the architecture of a frontier model on your own desk is now within reach. That does not mean anyone has built a smart conversational AI, though. Here is the full picture.
What it means for Japanese companies: Most companies won't be affected. It is relevant mainly to IT departments and training managers who want to develop engineers in-house who understand how AI works. Those companies may want to check this month whether the project runs to completion on a Mac with around 24GB of memory or a PC with an AMD GPU. It has no effect on contracts for Copilot, ChatGPT Enterprise or similar services.
What happened: a mini DeepSeek in 200 lines
The project is the work of solo developer Vaibhaav Tiwari, who published the first code for deepseek-v3-jax on November 13, 2026. JAX is a Python library from Google for numerical computing and machine learning. Leaving out blank lines and comments, the core code is 190 lines long, and it contains four of DeepSeek V3's signature mechanisms, explained in the next section.
The README gives specific training details. The machine was a MacBook Air M5 with 24GB of memory, running on Apple's framework for GPU access. The training data was 207 public-domain detective novels, which come to 21.69 million tokens, the small units an AI uses to read text. The model trained for 12,000 steps on about 98 million tokens in total, which means it read the same books about 4.5 times. Median throughput was about 10,700 tokens per second. At about 2.6 hours, a run can start before bed and be finished by morning.
Validation loss shows how much the model learned. It measures how often the model fails to predict the next word in text it never saw during training, and lower is better. The loss fell from 5.02 at step 250 to 3.51 at the end. Given the prompt "The fog came in off the," the model reportedly continued with "the bay, and the great trees were spreading and the stars were shining." The grammar holds up, and the line has a slight detective-novel feel.
The model has 36.2 million parameters, the adjustable values inside an AI, but only 13.6 million are active for each token. All 39 tests pass, and the trained weights file is 138MB. As of November 16, the repository had 199 stars and 17 forks on GitHub.
The four mechanisms inside
This is the most interesting part. The techniques credited with making the full DeepSeek V3 cheap and fast have been faithfully reproduced at small scale.
The first is MLA (Multi-head Latent Attention). When an AI writes text, it carries information about earlier words as a cache. MLA stores that cache in compressed form. In this repository, the cache per token per layer shrinks from 480 to 144, roughly 30% of its original size. The longer the text, the more memory this saves.
The second is DeepSeekMoE. The model has one shared expert that is always used and 16 routed experts, with the top four of the 16 selected for each token. That is why only 13.6 million of the model's 36.2 million parameters are active per token.
The third is auxiliary-loss-free load balancing. MoE models commonly suffer from work piling up on a few experts. The usual fix was an extra penalty term, the auxiliary loss, for uneven routing. DeepSeek V3 instead raises or lowers a per-expert bias that makes each expert more or less likely to be chosen. The effect is clear in the numbers. Without the bias, the share of tokens handled by each expert ranged from 0.09% to 18.6%. With it, the range narrowed to 5.7% to 6.9%. A perfectly even split would be 6.25%, so the result is close to ideal.
The fourth is MTP (multi-token prediction). Most models train only to predict the next token. MTP also has the model predict the token after that. Each training step gives the model more signal, which is said to make training more efficient.
Readers of a paper often skim past mechanisms like these. In our view, the project's main value is that you can run all four on your own laptop and watch the numbers change.
Another user completed a run on an AMD consumer GPU
Personal repositories often run only on their author's machine. This one already has a report from someone else who ran it, posted in a Show HN thread on Hacker News (a post for showing off a personal project) on November 14.
The report came from HN user joebarbere, who used an AMD consumer GPU, the RX 7600 XT (16GB). joebarbere called it the first project that had run from start to finish on that GPU. The README's AMD section lists the figures. Training finished in about 50 minutes, compared with about 2.6 hours on the Mac. Throughput was about 35,000 tokens per second (Mac: about 10,700), and final validation loss was 3.527 (Mac: 3.507). That makes it about three times as fast as the Mac, with nearly identical results.
Getting there took some work. The GPU was recognized only after an administrator-level setting was overridden, and a separate setting was needed to stop the GPU from hanging. joebarbere bundled these fixes into the project's first pull request, which the author merged on November 15. The commit log lists joebarbere and Anthropic's Claude as co-authors, a sign of the times. The README's AMD section has since been updated with the roughly 3x speed result.
As of November 16, the thread had 272 points and 27 comments. The project was also reportedly featured in Agent Tools Today (November 15, 2026 edition), a daily roundup of agent development tools, along with the author's headline figure: 36.2 million parameters pretrained on a MacBook Air in about 2.6 hours. When the roundup was written, the thread reportedly had 199 points and the repository 129 stars, so both have grown since.
What it can't do, and what isn't working
Here are all the limitations. To begin with, the author states plainly in the README that this is a demonstration of the mechanisms and not a strong language model. The full model has 671 billion parameters and this one has 36.2 million, about 1/20,000th the size. It has 6 layers to the original's 61, and its context window is just 256 tokens, compared with 128,000. It cannot handle work tasks such as summarizing business documents.
The internals are simplified as well.
- The MoE computes all 16 experts for every token and simply zeroes out the ones that weren't selected. It has no sparse computation that runs only the experts needed, and no mechanism for spreading experts across multiple GPUs
- It also leaves out the optimizations that make MLA cheaper to compute, FP8 training (low-precision training for speed), DualPipe (which reduces idle time between GPUs), YaRN (which extends the context length) and post-training
- The load-balancing bias evened out routing cleanly, but validation loss at step 2,000 was 3.921 with the bias and 3.923 without, essentially the same. At this scale, it shows no visible effect on accuracy
- Commenters on HN pointed out that the custom detective-novel dataset makes it hard to compare results directly with other implementations. The author replied that the data was chosen for fun
- The AMD GPU required overriding settings. One HN commenter said that after all these years, AMD's software support for anything other than its data-center hardware is still terrible
One more important caution: the distributed trained weights use a format that can run code embedded in the file just by loading it, and the README itself warns about this. If you try the project, train the weights yourself or confirm where the file came from first.
A few things could not be verified. The small CPU-only configuration has been checked by the author for only a few steps and has not been trained to completion. A question on HN about CPU-only performance has not yet been answered. As of November 16, joebarbere is the only person other than the author to report a complete run. With 199 stars, the project is still too new for a settled verdict.
Even so, we recommend it because it lets you go beyond reading about how frontier models are built and actually run one on your own desk. Remove the bias and routing imbalance jumps to 18.6%. The cache shrinks to 30% of its size. On AMD, a run takes 50 minutes. Numbers like these are yours with one laptop and one night. Anyone whose job involves explaining how AI works should take a look at the README.
Editorial cartoon
