Kubernetes and the Container Runtime Interface
Part 3: How Orchestration Plugs Into the Runtime Stack
Kubernetes doesn’t run containers. It schedules them. The actual running happens in containerd and runc layers you now understand.
In Part 1, you learned where containers exist: the Linux kernel.
In Part 2, you learned who talks to the kernel: runc calls syscalls, containerd manages lifecycle, Docker wraps it for developers.
In Part 3, we add the orchestration layer. How Kubernetes schedules workloads across many nodes. How it talks to container runtimes through a standard interface. Why it removed Docker support. What actually happens when you deploy a pod.
The complete stack:
┌─────────────────────┐
│ Kubernetes │ ← Part 3 (this article)
├─────────────────────┤
│ containerd │ ← Part 2 (covered)
├─────────────────────┤
│ runc │ ← Part 2 (covered)
├─────────────────────┤
│ Linux kernel │ ← Part 1 (covered)
└─────────────────────┘
Let’s start with the problem Kubernetes faced in 2016.
The Problem: Kubernetes Was Locked to Docker
Early Kubernetes (2014-2016) only supported Docker.
When you deployed a pod, Kubernetes talked directly to the Docker API. If you wanted to run containers in Kubernetes, you ran Docker.
This created problems:
1. Tight coupling
Kubernetes was tightly coupled to Docker’s API surface, creating maintenance overhead and limiting runtime flexibility.
2. No alternatives
Want to use rkt instead of Docker? Tough. Want to try a different runtime? Rewrite Kubernetes.
3. Feature creep
Docker kept adding features Kubernetes didn’t need. Docker’s networking, volumes, swarm mode Kubernetes already had its own.
4. Unnecessary layers
kubelet → Docker API → dockerd → containerd → runc → kernelDocker was a middleman. Kubernetes needed containerd, not Docker’s extra features.
The Kubernetes project needed a way to decouple from Docker while still supporting it. They needed a standard interface.
CRI: Container Runtime Interface
2016: Kubernetes introduces the Container Runtime Interface (CRI).
CRI is a gRPC API that defines how Kubernetes talks to container runtimes. It’s the boundary between orchestration and runtime.
The key insight: Kubernetes doesn’t care how you run containers. It just needs to tell a runtime: “Start this container with these settings.”
CRI standardizes that conversation.
The CRI API
CRI defines two gRPC services:
RuntimeService - Manage pods and containers
ImageService - Manage container images
That’s it. Any runtime that implements these two services can work with Kubernetes.
RuntimeService: Pod and Container Operations
RunPodSandbox - Create pod network namespace
StopPodSandbox - Stop all containers in pod
RemovePodSandbox - Clean up pod resources
CreateContainer - Create container in pod sandbox
StartContainer - Start the container
StopContainer - Stop the container
RemoveContainer - Delete the container
ListContainers - List running containers
ContainerStatus - Get container state
ExecSync - Run command in container
Exec - Attach to running processThese operations give Kubernetes everything it needs to manage pods.
ImageService: Image Operations
ListImages - Show available images
PullImage - Download image from registry
RemoveImage - Delete unused images
ImageStatus - Check if image exists locallySimple. Standard. Runtime-agnostic.
CRI Implementations
Any runtime that implements CRI can work with Kubernetes. Three main options:
1. containerd with CRI plugin
containerd has a built-in CRI plugin. Most popular choice.
kubelet → CRI gRPC → containerd CRI plugin → containerd → runc2. CRI-O
Purpose-built for Kubernetes. Implements CRI natively.
kubelet → CRI gRPC → CRI-O → runc3. Docker via dockershim (deprecated)
Kubernetes used to include a shim to make Docker CRI-compatible.
kubelet → dockershim → Docker API → dockerd → containerd → runcThis is what got removed. Let’s understand why.
The dockershim Story: Why Kubernetes Removed Docker Support
This caused massive confusion in 2020. “Kubernetes is deprecating Docker!” Headlines everywhere. Panic.
The reality: Kubernetes removed a compatibility layer, not container support.
What Was dockershim?
dockershim was code inside kubelet that translated CRI calls into Docker API calls.
Docker predates CRI. Docker’s API isn’t CRI-compatible. So Kubernetes maintained an adapter.
The problem with dockershim:
kubelet → dockershim (in kubelet) → Docker API → dockerd → containerd → runc
↑
Maintenance burden
↑
Extra layer
↑
Uses Docker, but Docker uses containerd anywayKubernetes maintainers had to keep dockershim updated with both CRI changes and Docker changes. It was technical debt.
Why Remove It?
1. Unnecessary indirection
Docker internally uses containerd. Why not use containerd directly?
With Docker: kubelet → dockershim → dockerd → containerd → runc
Without Docker: kubelet → containerd CRI plugin → runcSame underlying runtime. Fewer hops.
2. Maintenance burden
Kubernetes maintainers spent time maintaining dockershim instead of improving Kubernetes.
3. CRI exists
containerd and CRI-O implement CRI natively. No adapter needed.
4. Docker features unused
Docker’s image building, volumes, networking Kubernetes doesn’t use any of it. It has its own implementations.
What Actually Changed
Kubernetes 1.20 (December 2020): dockershim deprecated
Kubernetes 1.24 (April 2022): dockershim removed
Before removal:
kubelet → dockershim → dockerd → containerd → runcAfter removal:
kubelet → containerd (CRI plugin) → runcWhat stayed the same:
Your container images (OCI standard)
How containers run (same kernel features)
How pods work (same abstractions)
containerd (was always the real runtime)
What changed:
Can’t use dockerd as the runtime
Must use CRI-compatible runtime
For End Users: Nothing Changed
If you’re using managed Kubernetes (EKS, GKE, AKS):
Provider switched to containerd
Your pods still run
Your images still work
You noticed nothing
If you’re running your own cluster:
Update container runtime to containerd or CRI-O
Everything else stays the same
The “deprecation” was removing a compatibility layer, not removing container support.
Modern Kubernetes Runtime Architecture
Here’s the current architecture most clusters use:
┌─────────────────────────────────────────────────┐
│ Kubernetes Control Plane │
│ ┌───────────────────────────────────────────┐ │
│ │ API Server - Stores cluster state │ │
│ │ Scheduler - Assigns pods to nodes │ │
│ │ Controllers - Maintain desired state │ │
│ └───────────────────────────────────────────┘ │
└─────────────────────────────────────────────────┘
↓
schedules pod to node
↓
┌─────────────────────────────────────────────────┐
│ Worker Node │
│ ┌───────────────────────────────────────────┐ │
│ │ kubelet │ │
│ │ (Kubernetes agent on each node) │ │
│ └────────────────┬──────────────────────────┘ │
│ ↓ │
│ CRI gRPC │
│ ↓ │
│ ┌───────────────────────────────────────────┐ │
│ │ containerd (with CRI plugin) │ │
│ │ - Pulls images │ │
│ │ - Creates pod network namespace │ │
│ │ - Creates containers │ │
│ └────────────────┬──────────────────────────┘ │
│ ↓ │
│ OCI Runtime Spec │
│ ↓ │
│ ┌───────────────────────────────────────────┐ │
│ │ runc │ │
│ │ - Creates namespaces/cgroups │ │
│ │ - Executes container process │ │
│ └────────────────┬──────────────────────────┘ │
│ ↓ │
│ Linux kernel │
└─────────────────────────────────────────────────┘Each component’s role:
API Server: Central state store. Receives pod definitions, stores in etcd.
Scheduler: Watches for unscheduled pods, assigns them to nodes based on resources and constraints.
kubelet: Agent on each node. Watches API server for pods assigned to its node. Tells container runtime to start them.
containerd: Implements CRI. Pulls images, creates pod sandboxes, starts containers.
runc: Makes kernel syscalls to create containers.
kernel: Does the actual isolation work.
The CRI Boundary
The critical interface is between kubelet and containerd:
kubelet (Kubernetes-specific)
↕
CRI gRPC (standard interface)
↕
containerd (runtime-specific)This boundary lets you swap runtimes without changing Kubernetes:
Want containerd? ✓
Want CRI-O? ✓
Want kata-runtime for VM isolation? ✓
As long as it implements CRI, it works.
What Happens When You Deploy a Pod
Let’s trace the complete flow from kubectl apply to running containers.
Starting point:
kubectl apply -f pod.yamlWhere:
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
- name: sidecar
image: busybox:latest
command: ["sleep", "3600"]A pod with two containers sharing a network namespace.
Phase 1: API Server (Control Plane)
User runs: kubectl apply -f pod.yaml
kubectl sends HTTPS POST to API Server:
POST /api/v1/namespaces/default/pods
Body: [pod spec]
API Server authenticates, validates, runs admission controllers,
and persists the pod to etcd.
Pod now exists in cluster state.
Status: Pending (not scheduled yet)At this point: Pod is just data in etcd. Nothing is running.
Phase 2: Scheduler (Control Plane)
Scheduler watches API Server for unscheduled pods
Scheduler sees nginx-pod with no node assignment
Scheduler filters nodes based on constraints and scores
the remaining candidates to select the best node.
Scheduler picks: worker-node-1
Scheduler updates pod in API Server:
spec.nodeName: worker-node-1
Phase 3: kubelet Discovers Pod (Worker Node)
kubelet on worker-node-1 watches API Server
kubelet sees nginx-pod assigned to this node
kubelet validates pod spec
kubelet begins pod creation sequencePhase 4: Create Pod Sandbox
A pod sandbox is a shared environment for all containers in the pod. Primarily: a network namespace.
kubelet calls CRI RuntimeService:
RunPodSandbox(PodSandboxConfig{
metadata: nginx-pod,
hostname: nginx-pod,
log_directory: /var/log/pods/nginx-pod,
dns_config: {...}
})
containerd receives request
containerd creates pod sandbox:
1. Creates network namespace
unshare(CLONE_NEWNET)
2. Calls CNI plugins to set up networking
- Creates veth pair
- Assigns IP address (e.g., 10.244.1.5)
- Sets up routing
- Configures DNS
3. Creates pause container
- Minimal container that just sleeps
- Holds the network namespace open
- Runs as PID 1 in pod's PID namespace
containerd returns sandbox_id to kubeletWhat’s the pause container?
The pause container is a tiny container (∼750KB) that does nothing but sleep. It exists to keep the pod’s namespaces alive.
If all app containers in a pod stop, the pause container keeps the network namespace alive so containers can be restarted without losing the pod’s IP address.
Pod sandbox:
└─ pause container (keeps namespaces alive)
- Network namespace with IP 10.244.1.5
- IPC namespace
- PID namespace (optional)
At this point: Pod network is ready. IP assigned. No application containers yet.
Phase 5: Pull Images
For each container in the pod spec:
Container 1: nginx
kubelet calls CRI ImageService:
PullImage("nginx:latest")
containerd:
1. Resolves nginx:latest to specific digest
nginx:latest → nginx@sha256:abc123...
2. Checks if image exists locally
- If yes, skip download
- If no, continue
3. Downloads from registry
- Pulls each layer
- Verifies checksums
- Stores in content store
4. Returns image reference
Container 2: busybox
kubelet calls CRI ImageService:
PullImage("busybox:latest")
containerd downloads busybox imagePhase 6: Create Containers
For each container:
Container 1: nginx
kubelet calls CRI RuntimeService:
CreateContainer(
pod_sandbox_id: <from phase 4>,
config: {
metadata: {name: "nginx"},
image: "nginx:latest",
command: [...],
args: [...],
working_dir: "/",
envs: [...],
mounts: [...],
linux: {
resources: {
memory_limit_in_bytes: 536870912,
cpu_quota: 50000
}
}
}
)
containerd:
1. Creates snapshot from image layers
- Uses overlay2 driver
- Stacks nginx image layers
- Creates read-write top layer
2. Generates OCI config.json
{
"process": {
"args": ["nginx", "-g", "daemon off;"]
},
"root": {"path": "/path/to/snapshot"},
"linux": {
"namespaces": [
{"type": "pid"},
{"type": "mount"},
{"type": "ipc"},
{"type": "uts"},
{"type": "network", "path": "/var/run/netns/<pod-sandbox>"}
],
"resources": {
"memory": {"limit": 536870912},
"cpu": {"quota": 50000}
}
}
}
3. Container created (not started)
4. Returns container_id
Container 2: busybox
kubelet calls CreateContainer for busybox
containerd creates busybox container
Both containers will join the same pod network namespacePhase 7: Start Containers
Container 1: nginx
kubelet calls CRI RuntimeService:
StartContainer(container_id)
containerd:
1. Spawns containerd-shim
2. Shim calls runc:
runc run <container-id>
3. runc makes kernel syscalls:
- clone(CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWIPC | CLONE_NEWUTS)
Creates new PID, mount, IPC, UTS namespaces
- setns() to join pod's network namespace
Container now shares network with pod sandbox
- Writes to /sys/fs/cgroup/ to set resource limits
- mount() to set up overlay filesystem
- pivot_root() to change root directory
- execve() to run nginx process
nginx process starts:
- Runs as PID 1 inside its container namespace
- Can see pod sandbox's pause container (if PID namespace shared)
- Uses pod's network namespace (same IP)
- Binds to port 80
Container 2: busybox
kubelet calls StartContainer for busybox
containerd → shim → runc creates second container
busybox joins same pod network namespace
Both containers can talk via localhostProcess tree on the worker node:
systemd
└─ kubelet
└─ containerd
└─ containerd-shim (for nginx-pod)
├─ pause (keeps namespaces alive)
├─ nginx (main app)
└─ busybox (sidecar)Network namespace view:
Pod Sandbox (10.244.1.5)
├─ pause container
├─ nginx container
└─ busybox container
All three share:
- Network namespace (same IP, same localhost)
- IPC namespace (can use shared memory)
Each has separate:
- PID namespace (unless shareProcessNamespace: true)
- Mount namespace (different filesystems)Phase 8: Status Updates
kubelet monitors containers via containerd
kubelet reports container status back to the API Server.
The pod transitions to Running.At this point: Pod is Running. Accessible at 10.244.1.5. Both containers alive.
The Complete Call Chain
kubectl apply
↓
API Server (validate, persist to etcd)
↓
Scheduler (assign to node)
↓
kubelet (watch API Server)
↓
┌──────────────────────────────────────┐
│ CRI: RunPodSandbox │
│ → containerd creates network ns │
│ → CNI plugins set up networking │
│ → pause container holds ns open │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ CRI: PullImage (for each container) │
│ → containerd pulls from registry │
│ → stores layers │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ CRI: CreateContainer (nginx) │
│ → containerd creates snapshot │
│ → generates OCI config │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ CRI: StartContainer (nginx) │
│ → containerd → shim → runc │
│ → runc creates namespaces/cgroups │
│ → runc joins pod network ns │
│ → runc execves nginx │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ CRI: CreateContainer (busybox) │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ CRI: StartContainer (busybox) │
│ → joins same network namespace │
└──────────────────────────────────────┘
↓
Pod RunningEvery CRI call is a gRPC request from kubelet to containerd. Every container creation goes through runc to the kernel.
Pod Sandboxes: How Containers Share Namespaces
The pod sandbox concept is fundamental to Kubernetes. It’s what makes pods different from standalone containers.
What Containers Share
Network namespace (always shared):
Same IP address
Same network interfaces
Same ports
Containers can talk via localhost
IPC namespace (always shared):
Can use shared memory
Can use semaphores
PID namespace (optional, via shareProcessNamespace: true):
Can see each other’s processes
Can send signals between containers
What Containers Don’t Share
Mount namespace (separate):
Each container has its own root filesystem
Each has its own volumes
UTS namespace (separate by default):
Can have different hostnames
Cgroups (separate):
Each container has its own resource limits
The Pause Container’s Role
Pod: nginx-pod
├─ pause (infrastructure container)
│ - Runs: /pause (just sleeps)
│ - Creates: network, IPC, PID namespaces
│ - Keeps: namespaces alive
│
├─ nginx (app container)
│ - Joins: pod's network namespace
│ - Has: own PID, mount namespaces
│
└─ busybox (sidecar)
- Joins: same network namespace as nginx
- Has: own PID, mount namespacesIf nginx crashes and restarts, it rejoins the same network namespace that pause is holding open. The pod’s IP address doesn’t change.
Example: Communication Within a Pod
# nginx listens on port 80
# busybox can reach it via localhost
# From busybox container:
$ wget -O - http://localhost:80
# This works because they share the network namespaceThis is why sidecar patterns work in Kubernetes. A logging sidecar can read nginx logs via shared volumes. A proxy sidecar can intercept traffic via localhost.
The Three-Part Mental Model
Let’s consolidate everything you’ve learned across all three parts.
Layer 1: Kernel (Part 1)
Where containers actually exist.
Namespaces: Isolation (what process sees)
PID, network, mount, IPC, UTS, user, cgroup
Cgroups: Resource limits (what process uses)
CPU, memory, block I/O, network, devices
Overlay filesystems: Layered storage
Copy-on-write, shared image layers
A container is: Process + namespaces + cgroups + filesystem
Layer 2: Runtime (Part 2)
Who tells the kernel what to create.
runc: OCI runtime
Reads OCI bundle (config.json + rootfs)
Makes syscalls (clone, mount, execve)
Creates container process
containerd: Lifecycle manager
Pulls images, stores layers
Creates snapshots (overlay filesystems)
Calls runc via shim
Tracks container state
Docker: Developer platform (optional)
Wraps containerd
Adds build, compose, friendly CLI
Great for local development
Not needed in production clusters
Standards:
OCI Image Spec (image format)
OCI Runtime Spec (how to run containers)
Layer 3: Orchestration (Part 3)
Who manages containers at scale.
Kubernetes:
Schedules pods across nodes
Maintains desired state
Service discovery, load balancing
Rolling updates, scaling
CRI: Standard interface between Kubernetes and runtimes
RuntimeService (pods, containers)
ImageService (image operations)
Pod sandbox:
Shared network namespace
Holds containers together
Pause container keeps it alive
kubelet:
Agent on each node
Talks to API Server
Calls containerd via CRI
Monitors container health
The Complete Stack
┌──────────────────────────────────────┐
│ kubectl (user) │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ API Server (control plane) │
│ Stores state, authenticates │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ Scheduler (control plane) │
│ Assigns pods to nodes │
└──────────────────────────────────────┘
↓
┌──────────────────────────────────────┐
│ kubelet (worker node) │
│ Watches assigned pods │
└──────────────────────────────────────┘
↓
CRI gRPC (standard)
↓
┌──────────────────────────────────────┐
│ containerd (worker node) │
│ Implements CRI, manages containers │
└──────────────────────────────────────┘
↓
OCI Runtime Spec (standard)
↓
┌──────────────────────────────────────┐
│ runc (worker node) │
│ Makes syscalls to kernel │
└──────────────────────────────────────┘
↓
syscalls
↓
┌──────────────────────────────────────┐
│ Linux kernel │
│ Creates isolation, enforces limits │
└──────────────────────────────────────┘The Two Critical Standards
OCI (Open Container Initiative):
Enables swapping runtimes at the low level
runc, crun, kata-runtime all implement OCI
CRI (Container Runtime Interface):
Enables swapping runtimes at the Kubernetes level
containerd, CRI-O both implement CRI
These standards create modularity. You can swap components without breaking the stack.
Why Understanding This Matters
1. Debug production issues
When a pod fails to start:
Check kubelet logs (orchestration layer)
Check containerd logs (runtime layer)
Check cgroups (kernel layer)
You know which layer to investigate.
2. Make architectural decisions
Need VM-level isolation? Use kata-runtime (implements OCI)
Want minimal footprint? Use CRI-O instead of containerd
Need Windows containers? Use containerd with different shim
Understanding the layers helps you choose components.
3. Evaluate new runtimes
New technologies fit into this stack:
gVisor - Sandboxed kernel (implements OCI)
Firecracker - Lightweight VMs (can implement CRI)
Wasm runtimes - WebAssembly containers (can implement OCI)
Understanding the architecture helps you evaluate new tools.
Key Takeaways
1. Kubernetes is an orchestrator, not a runtime.
It schedules workloads. containerd and runc actually run them.
2. CRI is the boundary between orchestration and runtime.
Any CRI-compatible runtime works with Kubernetes. containerd and CRI-O are the main options.
3. Removing dockershim eliminated an unnecessary layer.
Docker uses containerd internally. Kubernetes now uses containerd directly.
4. Pod sandboxes share namespaces between containers.
The pause container holds the network namespace open. All containers in a pod join it.
5. Every pod deployment flows through these layers:
kubectl → API Server → Scheduler → kubelet → CRI → containerd → runc → kernel
6. Standards enable modularity.
OCI at the runtime level. CRI at the orchestration level. Swap components without breaking the stack.
7. The kernel does the actual work.
Everything else is orchestration and management. Namespaces, cgroups, and overlay filesystems are where containers exist.
The Complete Stack
Part 1: Containers are kernel primitives (namespaces, cgroups, overlay filesystems).
Part 2: Runtimes orchestrate those primitives (runc, containerd, Docker).
Part 3: Kubernetes orchestrates runtimes at scale (CRI, kubelet, pod sandboxes).
You now understand the full container stack from kernel syscalls to Kubernetes orchestration. Each layer has a defined responsibility, and each boundary is standardized.
When you run kubectl apply -f pod.yaml:
API Server stores the desired state
Scheduler assigns the pod to a node
kubelet creates the pod sandbox (network namespace)
containerd pulls images and creates containers
runc makes clone(), setns(), and execve() syscalls
The kernel creates namespaces and enforces cgroups
Your containers run, sharing a network namespace
Hands-On Practice
Cement your understanding:
1. Inspect a running pod’s namespaces
# Get pod's main container PID
kubectl get pod nginx-pod -o json | jq '.status.containerStatuses[0].containerID'
# On the worker node:
sudo crictl inspect <container-id> | jq '.info.pid'
# Check its namespaces
ls -l /proc/<pid>/ns/
# Compare with pause container's namespaces
# They should share network namespace2. Watch pod creation in real-time
# Terminal 1: Watch kubelet logs
journalctl -u kubelet -f
# Terminal 2: Deploy a pod
kubectl apply -f pod.yaml
# See CRI calls in kubelet logs:
# - RunPodSandbox
# - PullImage
# - CreateContainer
# - StartContainer3. Explore containerd directly
# List pods (in Kubernetes namespace)
sudo crictl pods
# List containers
sudo crictl ps
# Inspect a container
sudo crictl inspect <container-id>
# You're seeing what kubelet sees via CRI4. Compare Docker and Kubernetes container setups
# Docker container
docker run -d nginx
docker inspect <container-id> | jq '.State.Pid'
sudo ls -l /proc/<pid>/ns/
# Kubernetes pod
kubectl run test-pod --image=nginx
# Get PID and check namespaces
# Notice the difference:
# - Docker: one container, no pause
# - Kubernetes: pod sandbox + container5. Trace CRI calls with crictl
# Create pod sandbox manually
cat > sandbox-config.json <<EOF
{
"metadata": {
"name": "test-sandbox",
"namespace": "default",
"uid": "test-uid"
},
"linux": {}
}
EOF
sudo crictl runp sandbox-config.json
# List sandboxes
sudo crictl pods
# You just did what kubelet doesFurther Reading
Kubernetes Architecture:
Container Runtimes:
Deep Dives:
Alternative Approaches:
kata-runtime - VM-based container isolation
gVisor - Application kernel for containers
Firecracker - Lightweight virtualization
Final Thoughts
Three parts. One coherent story.
Part 1: Containers are kernel features (namespaces, cgroups, overlay filesystems).
Part 2: Runtimes orchestrate those features (runc, containerd, Docker).
Part 3: Kubernetes orchestrates runtimes at scale (CRI, kubelet, pod sandboxes).
You started with “containers are just processes.” You now understand exactly how those processes are created, isolated, limited, and orchestrated across distributed systems.
The container ecosystem is a well-architected stack with clear boundaries at each layer.
Questions? Corrections? You’ve completed the full trilogy. If anything isn’t clear, that’s on me let me know.



