One of the largest US open AI models arrives as promised, but needs more than 500GB of memory to run

Thinking Machines Lab, led by former OpenAI CTO Mira Murati, has released the weights (the model itself) of its AI model Inkling 2 Small. In its November 5 announcement, the company said it would release them under Apache 2.0 the following week, after external safety testing was finished, and it has done so. Apache 2.0 is a permissive license that allows commercial use, modification and redistribution. With 594B (594 billion) total parameters, the model is among the largest open models released by a US lab. Even in NVFP4, a format that compresses it to 4 bits, it needs more than 500GB of memory to run. This is not a model that anyone can easily try out on their own hardware.

For Japanese companies: Most companies will see no immediate impact. The release matters to IT departments in finance, manufacturing and local government that cannot send data outside the organization and want to run AI on their own infrastructure. Before setting budgets for the rest of this fiscal year, check whether your GPU servers have more than 500GB of memory combined. If they don't, compare the cost of renting capacity in the cloud with Tinker's pricing (about ¥48 per million input tokens and about ¥160 per million output tokens). Copilot, ChatGPT enterprise plans and Gemini contracts are not affected.

What was released

Thinking Machines released Inkling 2 Small on November 5. At that point it was available only through the company's Tinker service, and the weights had not been published. When Hugging Face was checked on November 12, there was still no official repository, only versions modified by third parties. With this release, anyone can now download the official weights from the company.

The specifications match the November 5 announcement (primary source).

・Scale: 594B total parameters. It is a mixture-of-experts (MoE) model that activates only 17B per inference pass, plus a 292M vision component. It handles text and images but not audio. Its context window is 256K tokens (1 million tokens in evaluation).
・Scores (company-reported): 31 on the overall index from third-party Artificial Analysis, 18 points higher than the earlier Inkling Small. 86.3% on the SWE-Bench Pro v2 (Hard) coding benchmark, 72.1% on DeepSWE v1.1 and 81.6% on the MMMU Pro image-understanding benchmark.
・Training: Pre-training was done in NVFP4 from the start. The model was trained on 8.4T tokens over 68 hours on NVIDIA GB300 NVL72 systems.

The choice of Apache 2.0 matters. Many other large open models come with conditions in their own licenses. Meta's Muse Spark Open, Moonshot's Kimi K3.1 Pro and Mistral's Large 4 require a separate agreement when companies with large revenues offer the model as a service. Apache 2.0 has no such revenue conditions. For companies that build the model into products they sell, the legal review becomes much simpler.

What "among the largest in the US" means

By size alone, it is not the largest open US model. Meta's Muse Spark Open, released on October 15, is bigger at 816B total. But Muse uses Meta's own license, and companies with more than $20 million in revenue over 12 months need a separate agreement with Meta to offer it as a service.

Among US models released under Apache 2.0, it surpasses Reflection AI's Beam (501B total, October 16). It is the largest known US model that can be used without revenue conditions.

The gap with Chinese labs remains, however. Moonshot's Kimi K3.1 Pro has about 2.8T total parameters, and Meituan's LongCat-2.5 has about 1.6T. LongCat-2.5 is released under the MIT license. US labs have only now pushed the size of models released under permissive licenses past 500B.

The 500GB-plus barrier

Because the model uses only 17B parameters per pass, it is light in terms of how fast it produces answers. Even so, all 594B parameters must be held in memory, because which parts are used changes with each query.

NVFP4 is a format that shrinks weights to 4 bits, and Inkling 2 Small was trained in this format from the start. Even so, running it requires more than 500GB of memory. NVIDIA's DGX Spark, a small machine for running AI locally, has 128GB of memory, so simple division puts the requirement at more than four units. For reference, Beam (501B total) was said to fit on a single server with eight H100s. Inkling 2 Small likely calls for similar data-center-class hardware.

For most companies, that leaves two realistic options.

・Use Tinker: $0.30 (about ¥48) per million input tokens and $1.00 (about ¥160) per million output tokens.
・Rent GPU servers in the cloud and run it yourself: Data does not have to be handed to an outside company, but it costs money to rent the servers and staff to run them.

This is not a model for small and midsize companies to try on local computers. Running it on your own infrastructure makes sense for companies whose heavy usage would make API fees expensive, or companies that cannot send data outside because of regulations or contracts.

A promise kept, and what can't be controlled

Few models announced with a promise of open weights have shipped on schedule. Tencent said on November 4 that Hy4 would be released "within days," but as of November 12 no official weights had appeared. Alibaba has offered Qwen4 only as a preview inside some of its services. Thinking Machines delivered on its promise about a week after the announcement.

On safety, the company wrote in its November 5 announcement that it had run chemical, biological, radiological and nuclear (CBRN) evaluations and FORTRESS, an evaluation that measures how often a model refuses requests for misuse. It also said it had put in place two layers of monitoring: real-time checks on tool calls and later review of activity logs. However, this monitoring runs inside the company's own service. It does not apply to people who download the weights and run them on their own infrastructure. Weights cannot be withdrawn once released. Every open model faces the same issue.

What could not be verified

As of this writing, the following points could not be verified: the exact date and time the weights became publicly visible; who conducted the external safety testing before release and what the results were; the breakdown of the more-than-500GB memory requirement (how much is for the weights and how much is working memory for long inputs); and whether versions in other formats, such as 16-bit, are available. In addition, the Artificial Analysis score of 31 is on the new index revised on October 27 and cannot be compared with scores other models received before that.

Sources: Thinking Machines announcement, Hugging Face model page