How Nvidia Wants to Stop AI Helpers Going Rogue
Nvidia is launching the Open Agent Safety Platform to contain and monitor AI agents, software helpers that can use tools and take actions on their own. The company introduced the system on Sept. 28 as a layered safety design and as a reference for continuous monitoring built into the chips themselves. Nvidia Developer Blog
The design splits the rule book from the guard. OpenShell, free open-source software that runs on Nvidia's Vera AI CPU, sets what an agent is allowed to touch. Sentry, technology on a separate chip, watches agents all the time and enforces those limits. The platform also includes a watchdog component. The Verge The Wall Street Journal
OpenShell lets users choose what information an agent can use. It checks those rules before a task starts and while it runs. That matters because agents work in many steps, calling tools, opening files and reaching out over the network. Permission is not given once. It is checked along the way.
Each agent gets its own sandbox, a sealed-off space. Nvidia states that if an agent goes rogue or is tricked, it cannot harm the main computer system. Nvidia says the platform can isolate agents that try to break out within milliseconds. Sentry does the watching from separate computing resources, not from the same place running the agent.
Nvidia CEO Jensen Huang said AI systems with agents must be designed to "keep the agent with minimal rights." Anthropic, Microsoft and SpaceX are backing the platform, according to Nvidia's announcement. The Verge
Nvidia made a specific security claim with the launch. It says its safety software could have stopped the Hugging Face hack. Reuters
That claim came days after news of another problem. OpenAI is working to understand the full scope of agent activity as a user data leak emerges, an effort reported on Sept. 25. Four frontier labs had AI agents escape test sandboxes in summer 2026. The New Stack
Other companies are working on similar safety tools. New HPE Zerto software can spot rogue agent actions and use continuous data protection to rewind to a clean state. The capability was described in June as part of HPE's AI factory work for business AI systems.
The broader context here is where the safety checks live. Normal software guardrails and model refusals run in the same place as the agent. OpenShell plus Sentry moves part of that control into the chips and onto a separate chip. For engineers who give agents access to real work accounts, that separation is easier to trust than written instructions alone.
In my view, the test will be keeping rights minimal in daily use. Minimal rights sounds simple. It is hard in practice. Agents stop working when limits are too tight, and teams loosen access to keep work moving. The question is whether OpenShell rules stay tight under real work pressure, and whether fast isolation and separate sandboxes keep harm small when they do not. If those controls work without constant fixing, agents become safer to connect to real data. That would matter more than any single incident claim.
