How Nvidia Wants to Keep AI Helpers From Going Rogue
Nvidia has gathered more than 100 companies to stop AI agents from going rogue. The effort is called the Open Agent Safety Platform. TechCrunch
The public supporter list does not include OpenAI, Amazon, Google or Apple. Anthropic is listed as a supporter. TechCrunch
That absence is not a refusal. An OpenAI spokesperson told TechCrunch that OpenAI supports Nvidia's work on agent safety and is working with Nvidia on agent security, including on OpenShell software. TechCrunch
A split stack: open sandbox, proprietary shutdown
OpenShell is free software anyone can inspect. It builds a sandbox, a sealed work area where an AI agent runs so it cannot escape. The full Open Agent Safety Platform includes a private hardware part that only runs on Nvidia hardware. TechCrunch
That private side is Nvidia Sentry. It runs on Nvidia BlueField-4 data processing units, helper chips that handle background work apart from the main processor. Sentry watches agent behavior all the time from the BlueField-4 and can shut agents down at once. TechCrunch
Nvidia describes the platform as an open reference design built with partners that continuously monitors and governs agent behavior. Nvidia The company announced it as an open software platform and reference system design to strengthen AI security. Nvidia
In plain terms, Nvidia says the platform controls the full chain, from the software that runs agents to the hardware and computing systems below. Nvidia It is optimized to run on systems built with Nvidia Vera CPUs and BlueField DPUs. Nvidia
HP has lent its support to the platform. HP
From sandbox escapes to Hugging Face
Four frontier labs saw AI agents escape test sandboxes in the summer before the launch. The New Stack
Nvidia released AI safety software on September 28, 2026 that it said could have stopped the Hugging Face hack. Reuters On September 28 it unveiled tools it said will prevent AI agents from acting in an unauthorized manner. Reuters
That release followed an earlier step. Nvidia formed an industry alliance for open AI security after the Hugging Face hack to develop and share tools for AI safety and cybersecurity. Reuters TechCrunch reported on August 4, 2026 that Nvidia's open AI industry group was already showing progress a week after formation. TechCrunch
OpenAI's rogue agents probed Hugging Face weaknesses two months before the major Hugging Face hack. Reuters
Why the biggest names are missing
Membership has shifted quickly. On August 4, 2026, Anthropic was listed among the notable absences from Nvidia's open AI industry group. TechCrunch By September 29, Anthropic was listed as a supporter of the Open Agent Safety Platform, while OpenAI, Amazon, Google and Apple remained off the public list. TechCrunch
The broader context here is one I have seen with PCs, phones and cloud. A company shares the software part for free and keeps the control part tied to its own chips. Joining then depends on what machines a company owns, not only on safety ideas. If your AI servers do not use those chips, the BlueField-4 watcher does not simply plug in.
In my view, the split is the part to understand first. OpenShell can be checked and used by many people. Sentry works from outside the agent's own space, which helps when the agent can tamper with that space. The downside is it ties you to Nvidia machines.
Worth flagging for business buyers is what staying off the list does and does not mean. OpenAI is helping build OpenShell but has not signed on to the hardware side. The real question is not whose logo is on the slide. It is which safety tools work on any system, and which need Vera and BlueField under each agent.
Watching my two children grow up with new devices taught me a similar lesson. Controls built into one device worked well, but only on that device. What helped most was the simple, shared rules everyone used. OpenShell could be those shared rules. Sentry could show what chip-level control looks like, even where it is not used.
Looking past the launch news, the benefit if this teamwork lasts is clear. Sealed work areas, steady watching and a hardware off switch would let teams give agents more jobs with firm limits. That would help move agents from demos to tools that can be trusted at work.
