Container security is a broad topic, often discussed alongside container images, runtimes, Docker, and Kubernetes. This article focuses specifically on container isolation, which is about keeping containers securely separated from other containers and from the host system they run on.
Containers are by their very nature multi-tenant – using a single instance of the software and shared infrastructure to serve multiple tenants (customers), resulting in lower costs, easier maintenance, and better scalability, but potentially sacrificing isolation and customization for individual tenants. Modern microservices run as containers on shared compute, using a shared host kernel. Shared resources demand isolation, not just for security, but also stability (a bug in an application shouldn’t be able to crash the entire system and take down every other microservice!)
OK, so WTF is container isolation? To answer that question, we have to start with the question “WTF is isolation?”
What is Isolation in Computing?
The concept of isolation predates multi-user operating systems like UNIX and Linux. The earliest isolation mechanisms relied on the classical perimeter model, keeping the outside world out and fully trusting the inside. For instance, the isolation of the 30-ton ENIAC computer, which calculated sensitive data for the hydrogen bomb in 1945, was achieved by locking it in a room with specialized access.
The development of multi-user operating systems like UNIX and the advent of computer networks around 1969 necessitated logical isolation. Now, an adversary could simply log in over the network instead of breaking into a physical room. While isolation is crucial for cybersecurity, it was initially developed for stability in shared operating systems, preventing accidental bugs from crashing other applications or the entire system.
Multi-user primitives like user accounts, filesystem permissions, and the root user were created in the 1970s for logical isolation and remain the basis of container isolation to this day.
UNIX Isolation: The Foundation of Containers
As Multics evolved into UNIX, most of the core philosophies around isolation were envisioned, and amazingly, these remain the bedrock of container isolation to this day. These include separating user accounts from the privileged root account, memory protection for applications and users, and isolating userspace from kernel space.
Userspace isolation in Linux
Userspace isolation is how Linux keeps processes separate on a machine. The Linux kernel implements these protections, and they are used to isolate containers on a system.
Key userspace isolation primitives include:
- Memory Isolation:
- Memory management units (MMU) keep memory segmented by application and restrict access to kernel memory from userspace.
- Kernel page table isolation (KPTI) keeps kernel memory isolated from userspace
- No execute (NX) and Data Execution Prevention (DEP) specify whether memory pages can be executable or writable, which mitigates several classes of memory corruption vulnerabilities
- Address space layout randomization (ASLR) - while not providing isolation, ASLR provides additional hardening by randomizing memory address ranges, thereby thwarting attacks that rely on well-known memory ranges to inject code
- Isolating Kernel Space from Userspace:
- System calls or syscalls are a stable API for kernel access from userspace and are the only way a process can request resources from the kernel
- Seccomp is a mechanism and policy engine that denies specific processes from making particular system calls to the kernel
- no_new_privs restricts a newly-started program's privileges that its parent did not have (such as setuid/setgid programs and file capabilities)
- Linux Security Modules (LSM) such as SELinux, AppArmor, Landlock, etc., that hook various subsystems and enforce access controls
- File Permissions: What most people think of as Linux security.
These components are the basis of container security with the exception of namespaces which we will discuss later on.
While Linux kernel isolation mechanisms are robust enough for a single-user system, it’s unsuitable for a multi-tenant container orchestration system like Kubernetes. This is because a user misconfiguration or a bug in the Linux kernel thwarts all of the protections highlighted here. The Linux kernel has become the general-purpose security boundary for the cloud, but it wasn’t purpose-built for secure container isolation.
How Container Isolation Works in Linux & Kubernetes
When containers were first introduced, they enabled the efficient execution of multiple workloads on commodity hardware, often at the cost of security. Container isolation is derived from the userspace primitives defined in UNIX and implemented in Linux. Containers are just Linux processes that depend entirely on the security of Linux namespaces, cgroups, and capabilities. In fact, the first Linux kernel patch even referred to them as "process containers," and every process on a modern Linux system runs inside a namespace.
- Namespaces create the illusion of isolation by segmenting a container's resources.
- PID creates a dedicated process ID table for containers.
- Network namespaces virtual network devices and routing tables.
- Mount mounts a containerized filesystem from a host directory.
- User is a newer namespace that remaps the root user inside a container to a non-root user on the host.
- Cgroups control the compute resources a container can consume to prevent a rogue workload from siphoning resources.
Containers were developed as a great way to run trusted workloads on trusted compute efficiently. However, while Linux kernel isolation mechanisms are robust enough for a single-user system, they are unsuitable for a multi-tenant orchestration system like Kubernetes. This is because a misconfiguration or a bug in the Linux kernel can thwart all these protections. The Linux kernel has become the general-purpose security boundary for the cloud, but it wasn't purpose-built for secure container isolation.
Virtualization Isolation: Stronger Than Containers
Virtualization provides a much stronger form of isolation than the Linux kernel. It fully isolates a guest operating system from the host hardware via a hypervisor. The hypervisor emulates hardware and completely segments memory from one guest to another, including isolating the Linux kernel, which provides the highest degree of isolation. This strength allows major cloud providers like AWS and GCP to safely host virtual machine guests from multiple, often competing, customers.
There are two kinds of hypervisors:
- Type-1 Hypervisors run directly on hardware for the highest performance and smallest attack surface. Examples include Xen and KVM.
- Xen has a minimal, microkernel design that is secure by default. It also runs with paravirtualization, meaning it offers strict isolation without specialized hardware requirements. This leads to a smaller Trusted Computing Base (TCB) compared to KVM.
- KVM's integration into the Linux kernel results in a significantly larger attack surface and TCB.
- Type-2 Hypervisors run on top of an existing operating system and inherit the attack surface of the host OS. Examples include Parallels and VirtualBox.
Hypervisors + Containers: ¿Por qué no los dos?
There are tradeoffs in every implementation, and containers are no exception.
Containers offer the most resource efficiency by binpacking, effectively processes, onto a machine at the cost of isolation and security.
The solution is to combine them. Lightweight and security-by-default hypervisors like Xen can efficiently run containers, combining VM-level security with container efficiency. This approach is how Edera enables secure multi-tenancy in Kubernetes for its customers.
Q1: What is container isolation?
A1: Container isolation is the separation of workloads to prevent one container from affecting others or the host system.
Q2: How does Linux isolate containers?
A2: Linux uses namespaces, cgroups, and capabilities to segment resources, but all containers still share the same kernel.
Q3: Why are hypervisors more secure than containers?
A3: Hypervisors isolate at the hardware level, preventing container escapes and ensuring stronger workload isolation.
Q4: Can containers and hypervisors be combined?
A4: Yes. Lightweight hypervisors like Xen combine VM-level security with container efficiency, enabling secure multi-tenancy in Kubernetes.
