Popular tool that compresses AI conversations hit by reports of sessions that no longer fit and stall in latest release

This is Haru. This time I looked into billion-context, a tool built by a solo developer that automatically compresses long exchanges with AI and claims to let "even a model with a small limit keep going for a month." In the same week that its GitHub star count jumped by about 2,500 in 7 days, people using the latest release, v0.1.213, have kept posting reports, with numbers attached, saying that "even after compression it didn't fit, and the conversation became unusable." The author's figures and the users' figures don't match. For this piece, I went through the published materials and user reports and set the two side by side.

For Japanese companies: Development teams and IT administrators trialing AI coding tools should check this week whether employees have installed this kind of "proxy" tool on their own. These tools can see everything sent to the AI, including text, code and API keys (the credentials used to access a service). Companies that don't use AI development tools internally are not directly affected.

What the tool is, and how much attention it's getting

First, why it matters. AI models can only read so much at once, so in a long task the oldest parts of the exchange eventually fall out of reach. billion-context runs as a proxy between an AI tool and the AI model, folding the accumulated conversation into several layers of summaries to keep it within the limit. The interesting part is that it lets the model itself decide when and what to fold. Instead of a person writing detailed rules, the AI judges that "a summary is enough for this part now."

It was created by ranxianglei, a personal account, and the repository was created on August 6, 2026. As of November 12, the GitHub repository had about 6.8k stars and 559 forks.

The growth has been steep. According to a review on andrew.ooo, a publication run by AI agents with human oversight, stars rose from 1,812 on October 18 to 6,569 on November 11, with about 2,500 added in the last 7 days alone, it reported. An automated daily report tracking development activity also said the project gained about 375 stars on November 11 alone, making it one of the day's main talking points.

The pace of updates also stands out. Releases reached v0.1.200 within 10 weeks, and the latest is v0.1.213, released on November 7. Simple division puts that at roughly 20 new versions a week. Fast fixes are appealing, but the flip side is that something that worked yesterday can break in today's version. Many of the current bug reports involve exactly this latest release.

The author's figures: separate measurements from estimates

The author has published records of 4.5 months of continuous use on three machines in the README and in a paper. This is primary-source material, so I'll present the numbers as given. Over 4.5 months there were 174,327 calls and 18.76 billion cumulative input tokens, or about 24.7 billion tokens across all three machines, the author says. About 90% of that, 17.01 billion tokens, were cache reads (reusing already-read content, which is cheaper and faster).

The author's main claim is that "no call ever exceeded the 200,000-token limit." Some sessions ran past 10,000 calls, with the longest at 8,584 and 12,049 calls. One example shows a history of about 900,000 tokens compressed into a summary of about 13,000 tokens. As a record of sustained long-term use, this much is convincing.

But some points should be kept separate when reading. The headline claim that "you need only one-fifth the tokens" is an estimate, not a measurement. The average context was 126,000 tokens, while the uncompressed case was estimated at 670,000 tokens, giving a calculated 5.3x. Moreover, that long record comes from a model that handles 1 million tokens, with an average context of about 120,000. With a model that has a 200,000-token window, the effect is said to be only 1.2x to 1.8x. That is far more modest than the phrase "a month even at 100K" suggests.

There is also a comparison experiment. Against the common approach of discarding the oldest parts first (the sliding-window method), billed tokens fell by 31% to 54% with no change in accuracy when recalling earlier parts of the conversation, the author says. However, the experiment was run on only one model, Qwen3.8-27B. The author also describes cases where the tool costs more: with windows smaller than about 64,000 tokens, the roughly 2K-token instruction text appended to every call makes it about 25% more expensive. For short conversations or small models, you're better off not using it.

The users' figures: conversations stalling on the latest release

Now for the user reports. The most specific is Issue #6277. A user who combined v0.1.213 with opencode, a separate AI coding tool, and branched a conversation saw the first exchange balloon to 1,738,557 tokens, 5.4 times the model's 320,000 limit. After 4 minutes and 11 seconds of compression, about 508,147 tokens remained, and the request was rejected with a 413 error (a signal that the payload is too large to accept). It was also judged non-retryable, and the conversation had to be deleted, the user said.

The user also wrote that a fix (PR #5511) had not yet been merged. When they applied a different workaround (#5509), the first exchange fit within 132,866 tokens, meaning it can fit depending on how it's built. The author had not yet responded to the November 11 post. So the author's record of "never exceeding the limit" and a user's record of "hitting 5.4 times the limit and stalling" sit side by side in the same week.

People combining it with other tools have kept reporting failures as well. Issue #6258 and #6259 report that the system prompt (the instructions given to the AI at the start) grows with every step, and once it exceeds 1,048,576 characters the entire session stops with a 400 error. #6164 says the "25% buffer" reserved for output is applied twice, leading to an unrecoverable 413 error. A user combining it with Codex CLI reported 29 rejected requests and six connection failures before the first response (#6172, #6171). On November 12, the author themselves filed a performance issue in which 13MB is rewritten on every save of less than 2KB, and immediately flagged it for merging.

In numbers, dozens of bug reports have been filed in the three days since v0.1.213 was released, about 27 of them from a single user. The most common involve stalled save operations, sessions stopping with 400 or 413 errors, and failures when combined with Codex or dsh. Some automated tests have also reportedly been failing since November 10. There are 623 open issues and 2,970 in total.

There are also Mac-specific problems. According to Issue #6257, the resident proxy keeps running after the app that launched it is closed. At that point it loses permission to access the local network, so every connection to a model hosted on the LAN fails with a "host unreachable" error, and it does not recover on its own. This bug hits hardest for people running models on internal servers, in an air-gapped local LLM setup, precisely to keep confidential data from leaving the company.

What to know before using it at work

The security concerns are also clear. The proxy sits in a position to see every prompt, all code and every API key sent to the AI. If an admin setting is used to accept external connections, there is no authentication. Auto-update is enabled by default and checks the npm package registry every three minutes. Because its contents can be swapped out without you noticing, I think workplaces that prioritize sandboxing to confine what agents can do, or running in closed networks, will find it hard to adopt as is.

On usability, andrew.ooo points out some concerns. The model does not unfold compressed content on its own, so a person has to ask when details are needed. That means a human has to keep track of "what the compression forgot." It's a tool you install so you can hand things off, yet it adds monitoring work, which honestly felt like a letdown to me.

Here is what I could not verify. I did not run the tool myself for this piece. The andrew.ooo review also says it couldn't test the tool itself because GitHub billing limits prevented it from running tests. It notes that all the figures come from the author's own paper, that the evidence is observational data from a single user (the author), and that the "5x" figure is an estimate. It is also not yet known whether the user-reported problems occur in other environments, or when the author will fix them.

Note that this is a personal tool, not an official product from a major company, so it has nothing to do with contracts or pricing for Copilot, ChatGPT Enterprise or Gemini. With that said, here is my conclusion. Answering the problem of "wanting to keep a conversation going longer" by having the AI fold it up itself is an interesting idea, and the author's 4.5-month record is real. But it pays off for long tasks on models with large windows, and actually costs more for short conversations or small models. And the latest release is in the middle of a wave of reports about stalled conversations. If you want to try it, do so in a personal environment where no confidential data flows, and wait until the bug reports settle down. Before deciding whether to install it on company devices, first confirm with your administrators what it can see and what it updates automatically.