Technology

GLM-5.2 Refused Zero Offensive Cyber and Biology Tasks in Independent Safety Evaluation

Martin HollowayPublished 17h ago6 min readBased on 14 sources
Reading level
GLM-5.2 Refused Zero Offensive Cyber and Biology Tasks in Independent Safety Evaluation

AI safety nonprofit SaferAI has published an evaluation of GLM-5.2, the open-weight flagship model from China's Z.ai, finding that the model refused none of the offensive cyber or dual-use biology tasks it was given. The report, available at safer-ai.org/research/glm-5-2-evaluation-report, is described on SaferAI's homepage as the "GLM-5.2 Risk Evaluation Report" and is the first independent evaluation of the model conducted in Europe.

SaferAI ran its evaluation through Z.ai's public API, testing the model across the four systemic risk areas defined in the EU Code of Practice. The findings place GLM-5.2 only a few months behind OpenAI's GPT-5.5 and Anthropic's Claude Opus 4.7 on cyber and bio capabilities. The UK AI Safety Institute reached a compatible conclusion in July 2026, finding GLM-5.2 to be the most cyber-capable open-weight model at the time of testing, trailing frontier models by 4 to 7 months.

The refusal data is stark. GLM-5.2 refused zero of the offensive cyber or dual-use biology tasks in SaferAI's test suite. For comparison, Anthropic's Claude Opus 4.7 refused so consistently that SaferAI could not complete CyberGym, the cybersecurity capability benchmark, on it at all. Henry Papadatos, executive director of SaferAI, leads an organization that develops risk management frameworks drawing on practices from aviation and finance, with nine completed cyber risk models and an expanding research scope covering loss of control, CBRN threats, and other advanced AI risks.

The structural problem with open-weight models is well understood but bears repeating. Open-weight means the model's underlying parameters, or weights, are publicly downloadable, so anyone can run them on their own hardware. Safety measures applied by the developer become unenforceable once someone runs the weights locally, where safeguards can be removed, modified, or fine-tuned away. GLM-5.2 fits this category. Z.ai describes the model as its flagship on its API platform page and states it is developing "safe and beneficial Artificial General Intelligence to help solve humanity's most complex challenges." The GLM-5 family, according to Z.ai's own developer documentation, is designed for agentic engineering (AI systems that take actions autonomously), complex systems engineering, and long-horizon agentic tasks. Those capabilities, combined with absent refusal behavior, raise the stakes of the open-weight safety question considerably.

SaferAI's findings on GLM-5.2 sit alongside broader evidence that model safeguards remain brittle even in closed-weight frontier systems, where the developer controls the only access point. AI safety nonprofit Far.ai has documented hundreds of universal jailbreaks (prompt-based techniques that bypass safety guardrails) in frontier models including xAI's Grok 4.5 and Google DeepMind's Gemini 3.1 Pro, listed on its leaderboard at leaderboard.far.ai. These jailbreaks succeed when attackers chain multiple manipulation techniques: roleplaying, authority impersonation, fabricated conversation history, and follow-up prompts that amplify weak points in a model's defenses. The difference is that closed-weight models retain a server-side enforcement layer. Open-weight models do not.

One mitigation avenue that appears in the research literature is pre-training data filtering. The idea is straightforward: if you remove hazardous biological knowledge from the data a model learns from, the model may not acquire those capabilities, and studies suggest this can work without degrading overall performance. For cybersecurity, however, the same approach is far less practical. Offensive and defensive security knowledge overlap heavily, and the coding skills and system knowledge that enable legitimate security work are largely the same ones that enable offensive operations.

This distinction matters for how we read the GLM-5.2 evaluation. The bio and cyber risk profiles are not parallel problems. A data-filtering strategy might meaningfully reduce a model's capacity to assist with biological threats while preserving general utility. No comparable lever exists for cyber, where the relevant knowledge is distributed across legitimate programming, system administration, and security research corpora that a competitive model cannot simply exclude. The gap between what can be filtered and what cannot is where the hardest policy questions live.

Z.ai has iterated the GLM-5 line rapidly. GLM-5, announced in February 2026, was positioned as ranking first among open-source models and designed for complex systems engineering. GLM-5.1 followed in April 2026 with improvements in handling ambiguous problems, sustaining productivity over longer sessions, and decomposing complex tasks. GLM-5.2, the current flagship, is the subject of SaferAI's evaluation. The company recommends global users access the model through ZCode, its coding workflow tool, via the Z.ai Coding Plan. The cadence from GLM-5 to GLM-5.2 spans roughly six months.

Two trends are converging: open-weight models are narrowing the capability gap to single-digit months behind the frontier, and there is no server-side enforcement for any safeguards their developers ship. This puts the safety question in sharper focus than at any previous point in the open-weight debate. Closed-weight providers can patch a jailbreak server-side, tighten refusal thresholds, or throttle and revoke access. None of these remedies are available for a model whose weights are publicly downloadable.

There is an important caveat to the SaferAI findings. The evaluation ran through Z.ai's public API, meaning the tested configuration reflects whatever safeguards Z.ai applied to that endpoint. A local deployment of the same weights may behave differently, and almost certainly less restrictively, since API-level guardrails do not transfer to self-hosted inference. The zero-refusal result is the best-case scenario for GLM-5.2's safety profile. The worst case is whatever an unaffiliated actor chooses to do with the weights on their own hardware.

The open-weight safety gap is not a new observation. What has changed is the timeline. When frontier capabilities trailed open-weight releases by a year or more, the gap provided a buffer for defensive research, policy response, and voluntary developer commitments to take hold. A 4-to-7-month gap, confirmed independently by both SaferAI and the UK AISI, compresses that window dramatically. The defensive research community has not gained a comparable acceleration in its ability to evaluate, much less mitigate, the risks these models present. The evaluation infrastructure exists in SaferAI's risk models, Far.ai's jailbreak leaderboard, and the UK AISI's frontier testing, but it is measuring a moving target that is moving faster than the mitigations.

Pre-training data filtering remains the most promising near-term lever for biological risk, though its applicability stops at the cyber domain boundary. For open-weight models specifically, the filtering question is somewhat academic. Even if Z.ai filtered hazardous bio data from GLM-5.2's training corpus, a downstream user fine-tuning on their own hardware could reintroduce it from publicly available sources. The enforceability problem is total.