SemiAnalysis's new GDPx benchmark: even top-ranked Opus 5.5 solves only 12%

SemiAnalysis, a U.S. research firm that covers semiconductors and AI infrastructure, has released GDPx, a third-party benchmark that measures what AI agents can do. It sets tasks in 11 professional fields and gives an agent up to eight hours for each one. Claude Opus 5.5 (Anthropic) took first place and still solved only 12.0% of them. There is a second important finding. In three fields, cyber, biology and alignment (research on getting AI to behave as people intend), the models' safety classifiers kicked in and lowered their scores. In other words, the answer to "which model is strongest" depends on which field you test and how the safety measures are set.

For Japanese companies: If a company picks an in-house AI tool based only on the overall rankings reported in the news, it may pick the wrong one. Companies that use AI for security reviews or R&D in particular may find that safety mechanisms stop the work and the tool does less than expected. Testing on the company's own real work before deciding is the safer approach.

Just over 10% solved, even with eight hours

GDPx does not test agents on short questions. It has them finish the kind of work that takes a professional hours. Each task allows up to eight hours. As the "GDP" in the name suggests, the tasks appear to be chosen to resemble work that creates real economic value.

The results were harsh. Opus 5.5 ranked first and still got only 12.0% right. Put another way, even the strongest model failed to finish 88% of the tasks. On other benchmarks, the top models all score high and are hard to tell apart. By making tasks longer and harder, GDPx shows in numbers that there is still a lot of room to improve.

The benchmark comes from SemiAnalysis, not from a company that builds models. That matters: these are not numbers the AI companies published about themselves, but results measured by an outside party under the same conditions.

Three fields where safety mechanisms lowered scores

The most striking results are in three fields: cyber, biology and alignment. There, scores fell not because the models lacked the ability, but because of safety classifiers. A safety classifier is a system that automatically spots requests that could be misused, such as attack code or dangerous biological information, and blocks them.

Even when a task is legitimate work, if it involves attack techniques or the properties of pathogens, the classifier cannot tell it apart from a dangerous request and blocks it. The model is then scored below what it can actually do. Alignment research has the same problem, because it includes work that reproduces misaligned AI behavior.

This is less a flaw than a trade-off the companies chose on purpose. They prevent misuse at the cost of turning down some legitimate professional work. GDPx makes that cost visible from the outside as a score.

Rankings depend on how you measure

The same thing is already happening in other third-party evaluations. On the overall index from Artificial Analysis, a third-party firm that compares AI models, Opus 5.5 took first place with 58 points at the end of October. The next day, Google's Gemini 4 Carbon passed it with 59. On Terminal-Bench 4.0, which measures terminal operation, Carbon scores 65.2%, ahead of Opus 5.5's 59.6%. Change the yardstick and the leader changes quickly.

Safety settings also move the rankings. On Artificial Analysis's cyber index, an OpenAI version available only to vetted organizations scored 32 points higher than a general-release model from the same family, because its safety measures do not block it. On October 30, OpenAI released GPT-Rosalind-6 for biology. It has the same weights as GPT-6 Astra, with no extra training. OpenAI only reduced how often it refuses potentially dangerous biology questions, and its scores on specialist tests went up sharply. Anthropic also has a program that gives vetted security professionals a version with fewer restrictions.

The GDPx results fit this trend. How "strong" a model is today does not depend only on its weights. What it can actually do depends on who it is released to and with what safety settings.

What buyers should look at

There are three takeaways.

First, overall rankings are only a rough guide. Sales materials, accounting checks, code fixes and security assessments may each call for a different model.

Second, it is still too early to hand off long jobs entirely. A score of just over 10% for the top model on eight-hour tasks shows that even when an agent does the work, people need to check it along the way.

Third, security and R&D teams should plan on safety mechanisms blocking some of their work. If needed, it is worth looking into the vetted access programs that the companies offer. The stronger a model's safety measures, the more likely it is to fail at legitimate specialist work. Choosing with that in mind is what 2026, the "year of evaluation," calls for.