Nvidia OpenShell is an Apache 2.0-licensed open source runtime designed to provide security, privacy, and operational guardrails for AI workloads
Artificial intelligence is gradually evolving from interactive chat interfaces into autonomous software capable of writing code, executing commands, accessing files, interacting with APIs, and making decisions during execution. As AI agents become increasingly capable, an important question emerges: how can they safely interact with Linux systems?
Linux already provides mature security mechanisms such as SELinux, AppArmor, seccomp, namespaces, capabilities, Linux Security Modules (LSMs), etc. These technologies remain the foundation of secure computing. However, AI agents introduce a different execution model. Their behaviour is influenced by prompts, tool outputs, intermediate results, and external inputs rather than being completely predetermined.
Nvidia OpenShell is an Apache 2.0-licensed open source runtime that does not replace existing Linux security mechanisms. Instead, it builds upon them to provide runtime controls tailored to AI workloads.
| Note: OpenShell is a relatively new open source project. This article focuses on the architecture and concepts described in the official Nvidia documentation available at the time of writing. |
Why runtime governance matters
Traditional applications generally exhibit predictable behaviour. A web server processes requests, a database stores records, and a backup utility performs predefined tasks. Their behaviour is largely determined by static application logic. AI agents behave differently. During a single session, an agent may inspect source code, execute shell commands, modify files, download dependencies, invoke language models, and communicate with external services. The exact sequence of operations depends on prompts, intermediate responses, and runtime context.
This raises several practical questions for infrastructure and development teams:
- Which directories should an agent access?
- Which commands should it be allowed to execute?
- Which external services should it contact?
- How should secrets and API credentials be protected?
- How can unintended actions or privilege escalation be prevented?
These concerns introduce the need for runtime governance. Linux provides a mature collection of security primitives refined over decades. SELinux and AppArmor implement mandatory access control. Seccomp reduces the kernel attack surface by restricting available system calls. Namespaces isolate processes and resources, while capabilities divide root privileges into smaller units to support least-privilege execution. Landlock extends the Linux Security Module framework and provides kernel-enforced file system restrictions. These mechanisms continue to form the foundation of secure Linux systems.
Containers and virtual machines provide strong workload isolation and remain indispensable components of modern infrastructure. However, they primarily focus on isolating processes and resources. AI agents introduce additional requirements. Infrastructure and development teams must control external connectivity, protect secrets from agent visibility, govern model access, and apply runtime policies to actions influenced by prompts and external inputs.
OpenShell addresses these concerns through runtime controls while relying on existing Linux isolation mechanisms underneath.
OpenShell architecture
OpenShell separates control-plane responsibilities from local runtime enforcement.
The ‘gateway’ acts as the control plane and is responsible for authentication, policy management, and sandbox lifecycle operations.
Agents execute inside isolated ‘sandbox’ environments. Within the sandbox, the ‘supervisor’ acts as the local enforcement point.
The ‘policy engine’ evaluates policies governing filesystem access, process execution, network communication, and inference behaviour.
The ‘privacy router’ helps isolate credentials and manages model interactions without exposing sensitive information directly to the agent environment.
The AI workload itself runs as a restricted process under the supervision of the local enforcement layer. Filesystem and process boundaries are established when the sandbox is created, while communication and inference policies remain logically separated from those boundaries.
Core protection layers
Filesystem layer
OpenShell uses Landlock to restrict filesystem visibility. Agents receive access only to explicitly allowed paths, with enforcement performed by the Linux kernel. Since these restrictions are enforced by the kernel, they remain effective regardless of application behaviour.
Process layer
OpenShell relies on standard Linux mechanisms such as seccomp (secure computing mode) filters and privilege reduction to minimise the attack surface and help prevent privilege escalation.
Network layer
Network policies determine which destinations agents are permitted to contact. OpenShell separates communication policies from filesystem boundaries established during sandbox creation, allowing network access to be governed independently.
Inference layer
Instead of exposing provider credentials directly to the agent process, inference requests are routed through privacy-aware interfaces. Credentials remain outside the agent environment, reducing the possibility of accidental disclosure.
OpenShell and Linux kernel primitives
OpenShell does not introduce a new kernel isolation mechanism. Instead, it acts as a user space framework built on established Linux security primitives.
The supervisor establishes security boundaries and launches the agent process inside those constraints.
| Mechanism | Primary purpose |
| SELinux | Label-based access control |
| AppArmor | Path-based access control |
| Seccomp | System call filtering |
| Namespaces | Resource isolation |
| Capabilities | Least-privilege execution |
| Landlock | Filesystem sandboxing |
| Containers/VMs | Workload isolation |
| Nvidia OpenShell | Runtime governance for AI agents |
Traditional Linux mechanisms focus primarily on processes and resources. OpenShell adds AI-specific runtime controls on top of those foundations rather than replacing them.
OpenShell should not be viewed as a replacement for SELinux, AppArmor, seccomp, namespaces, containers, or virtual machines. These mechanisms continue to provide the fundamental isolation boundaries of Linux systems. OpenShell operates one layer above them, providing runtime governance tailored to AI agents. In practice, it complements existing security primitives rather than replacing them.
Traditional Linux security mechanisms focus primarily on processes and resources. OpenShell addresses additional concerns such as controlled access to files, network communication, credential isolation, and interactions with model providers. It builds on top of existing operating system capabilities rather than introducing a new isolation model.
AI agents introduce a different execution model from traditional applications. Nvidia OpenShell builds upon established Linux security primitives rather than replacing them. By combining sandboxing, policy-based controls, privacy-aware inference routing, and kernel-enforced restrictions, it provides a framework for governing autonomous AI workloads while leveraging the proven foundations already available in Linux. Rather than reinventing isolation mechanisms, OpenShell demonstrates how existing Linux security capabilities can be combined with additional runtime controls to address the unique requirements of AI agents.
