OpenAI math review hub logs one withdrawal as dispute over prior-work citations emerges

Mathematicians have set up their own hub to re-examine the mass release of math manuscripts that OpenAI published on October 6. The hub is mathcommunity/oai-math-revised on GitHub, and its commit history now records the withdrawal of one OpenAI manuscript. In pull request #38, experts are also publicly debating whether one manuscript properly cites prior research. The hub was featured on Terence Tao's blog only on October 24. Even so, the question of how far the 719 manuscripts can be trusted is already being tested in concrete terms, one paper at a time.

What happened at the hub

Abhinav Nellore, Johannes Schmitt, Soledad Villar and Rachel Ward set up oai-math-revised. In a guest post on Tao's blog, the four wrote that the manuscripts OpenAI posted to github.com/openai/math were "hard to follow" and that their citations were "incomplete." They called on students, postdocs and professors to help with peer review and maintenance.

The two new developments show that this call has turned into actual work. The first is the withdrawal. On October 7, OpenAI had already withdrawn three manuscripts on its own, citing sign errors, which cut the release from 722 to 719. The latest withdrawal was recorded on the hub run by outside mathematicians and is separate from OpenAI's own retractions. The second is the debate in #38. That dispute is not about whether a proof is correct. It is about attribution: how much of the thinking in a manuscript is new, and how much comes from existing research.

Why citation is the sticking point

In mathematical papers, correctly stating where ideas came from is a basic norm, on a par with getting the proof right. Critics have raised this point again and again since OpenAI's release. On October 13, Sophie Huiberts, also writing a guest post on Tao's blog titled "This is normal bad work," judged the released manuscripts against professional norms and called them "normal bad work." Her reasons were improper attribution of where ideas came from, results that had been "salami-sliced," and proofs that were hard to decipher. In the third round of First Proof, whose results came out on October 14, reviewers also scored attribution of prior work, alongside correctness and writing.

#38 takes these general criticisms down to specific questions: which passage in a particular manuscript should have cited what. The criticism has moved from essays to the peer-review record.

What Lean cannot check

OpenAI says it checked about 42% of the manuscripts (about 300) in Lean. But formal verification guarantees only that a proof is logically correct. Lean's kernel cannot tell whose idea something is, or how closely a result retraces existing work. In an October 9 guest post, Thomas Hales questioned the reliability of the Lean kernel itself, but the current dispute lies even further outside what the kernel covers. A manuscript that is correct but not new, or that looks new because it fails to cite prior work, slips past machine verification.

The volume is also overwhelming. The 719 manuscripts, which make up 372 groups of results drawn from about 4,000 problems, have to be read one by one by mathematicians and checked against the literature in each field. This imbalance is why Andrej Karpathy, on October 11, cited these math manuscripts as an example of human review failing to keep up. The hub's call for help with peer review and maintenance also reflects how heavy that work is.

Open questions

One withdrawal and one attribution dispute are small numbers next to 719 manuscripts. But they surfaced as soon as outside mathematicians began reading in an organized way. Whether similar findings pile up as review continues, or stay rare, will decide how this mass release is judged.

OpenAI also has questions it has yet to answer. It still has not named the internal model that produced these results. A $15 million program launched on October 15 by the Simons Foundation, Anthropic and OpenAI supports building ways to evaluate AI-assisted results. What is happening at oai-math-revised is exactly that kind of evaluation. The next question is how the company that released the manuscripts responds to the peer-review process the math community is putting together.