The problem
Log4Shell (CVE-2021-44228) is remote code execution in the Log4j 2 logging library. It showed how little stands between a flaw in one library and a full foothold when a container runs with its defaults. Once an attacker can run code inside the application process, a default container usually allows the following:
- The process runs as
root(UID 0) inside the container. - The root filesystem is writable, so tools, miners or modified configuration can be dropped and kept.
- The process holds a set of Linux capabilities it never uses.
- In Kubernetes, every pod can open connections to every other pod, so a compromised frontend can reach a database directly.
Patching the vulnerable library is the first step. It does not limit what the next zero-day can do after it lands in the same kind of container. The aim here is to shrink that blast radius.
Why it happens
Container defaults favour compatibility over containment.
- Root user. An image without a
USERinstruction starts its process as UID 0. Unless user namespaces (userns-remap) or rootless mode are enabled, UID 0 in the container is also UID 0 on the host, restricted only by capabilities, seccomp and any AppArmor or SELinux profile. A runtime bug such as CVE-2019-5736, where a malicious process could overwrite the hostruncbinary, turns that into host root. - Writable layer. Docker adds a writable layer on top of the image unless told otherwise.
- Capabilities. Docker starts containers with a default set of capabilities. A common misconception is that this set includes powerful ones such as
CAP_SYS_MODULEorCAP_SYS_TIME. It does not, and those are only available with explicit--cap-addor--privileged. The default set still containsNET_RAW,DAC_OVERRIDE,SETUID,SETGIDandCHOWN, which most services do not need. - Flat pod network. A Kubernetes pod is non-isolated until a NetworkPolicy selects it, so by default all ingress and egress is allowed.
- Shared kernel. Containers on one node share one kernel. A kernel or runtime vulnerability crosses container boundaries, and a kernel panic takes down every container on the node.
Source of truth
The fixes below rely on these references:
- Docker run reference. It documents
--read-only,--tmpfs,--user,--cap-dropand--cap-add. It also lists the capabilities a container gets by default and shows that--cap-drop=ALLfollowed by selective--cap-addis the supported way to restrict them. - Dockerfile reference, USER. It confirms that
USERsets the user for the rest of the build stage and for the container's default process. - Kubernetes network policies. They confirm that pods are non-isolated by default and become isolated once a policy selects them. They also confirm that policies only take effect if the cluster's network plugin enforces them, and they show the default-deny pattern.
- Kubernetes security context task. It documents
runAsNonRoot,runAsUser,readOnlyRootFilesystem,allowPrivilegeEscalation,capabilitiesandseccompProfile. - NVD entry for CVE-2019-5736. It describes the runc container escape and the fixed versions.
- NVD entry for CVE-2021-44228. It describes the Log4j 2 flaw behind Log4Shell.
Fix options
| Area | Option | Trade-off |
|---|---|---|
| Process user | USER in the Dockerfile | Travels with the image, simple. Files must be readable by that user. |
| Process user | --user or Kubernetes runAsNonRoot | Enforced at deploy time even for images that forgot USER. |
| Process user | User namespaces or rootless Docker | Maps container root to an unprivileged host UID. Stronger, but has volume and networking limits. |
| Filesystem | --read-only plus --tmpfs or volumes | Blocks persistence. Needs the writable paths identified first. |
| Privileges | --cap-drop=ALL plus selective --cap-add | Least privilege. Needs testing, because some images need CHOWN, SETUID or SETGID at start-up. |
| Privileges | Drop only selected capabilities | Easier, but still allows everything not named. |
| Network | Default-deny NetworkPolicy plus allow rules | Needs a network plugin that enforces policy, and egress rules for DNS. |
| Isolation | Sandboxed runtimes (gVisor, Kata) or one VM per trust zone | Strongest kernel boundary, with extra overhead and complexity. |
Why this fix
The practical choice is to layer the cheap, well-documented controls. They address the conditions an attacker needs after code execution, and they do not conflict with each other:
- A non-root user in the image, enforced again at runtime. This removes the easiest route to a runtime escape and to file modification.
- A read-only root filesystem with explicit tmpfs or volume mounts. This removes the easy persistence path.
cap_drop: ALLwith only the capabilities the service demonstrably needs, plusno-new-privileges, so setuid binaries cannot raise privileges.- A default-deny NetworkPolicy with explicit allow rules, including DNS, so a compromised pod cannot reach arbitrary peers.
These controls need no new software, only configuration, and they are supported by Docker 20.10 and Kubernetes 1.19 or later. Sandboxed runtimes and per-VM isolation are reserved for workloads where the shared kernel is itself the unacceptable risk.
Step by step
The commands below are meant to be applied and checked on a test host. This article does not reproduce terminal output. The expected result of each check is described in prose.
1. Non-root image
The official Node.js images already ship an unprivileged node user (UID 1000). Using it avoids creating one. Leaving application files owned by root keeps them read-only to the process, which is better than a recursive chown to the app user.
FROM node:16-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
USER node
CMD ["node", "index.js"]
For images without a ready-made user, create one with addgroup -S appgroup && adduser -S appuser -G appgroup (Alpine) before the USER line.
2. Run with a read-only filesystem and no capabilities
docker build -t my-secure-image:latest .
docker run -d --name app \
--read-only \
--tmpfs /tmp \
--cap-drop ALL \
--security-opt no-new-privileges \
-p 8080:8080 \
my-secure-image:latest
Port 8080 is above 1023, so NET_BIND_SERVICE is not needed. A service that binds port 80 or 443 inside the container needs --cap-add NET_BIND_SERVICE. Stock nginx images also typically need CHOWN, SETGID and SETUID because the master process starts as root and drops to the worker user. Add them back one at a time if start-up fails with a permission error. Prefer writing logs to stdout and stderr, which avoids a writable log volume.
The same settings in Compose file format 3.x:
version: '3.8'
services:
app:
image: my-secure-image:latest
read_only: true
tmpfs:
- /tmp
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
ports:
- "8080:8080"
3. Verify
docker exec app id
docker exec app touch /app/test.txt
docker exec app touch /tmp/test.txt
docker inspect --format '{{.HostConfig.ReadonlyRootfs}} {{.HostConfig.CapDrop}}' app
id should report UID 1000 and not 0. The write to /app should fail with a read-only file system error. The write to /tmp should succeed, because it is a tmpfs. The inspect command should show the read-only flag as true and ALL in the dropped capabilities. To find what a service needs, start it with --cap-drop ALL in a test environment and watch for "Operation not permitted" errors in its logs. capsh --print inside a container lists the capability sets it actually holds.
4. The same controls in Kubernetes
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: app
image: my-secure-image:latest
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
5. Default-deny network policy, with DNS and one allowed path
Denying egress also blocks name resolution, so DNS has to be allowed explicitly. The kubernetes.io/metadata.name namespace label is set automatically from Kubernetes 1.21.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: default
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns-egress
namespace: default
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: frontend-to-backend
namespace: default
spec:
podSelector:
matchLabels:
app: backend
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
The frontend pods also need an egress rule to the backend, because default-deny covers both directions. Apply and inspect the policies with:
kubectl apply -f policies.yaml
kubectl get networkpolicy -n default
kubectl describe networkpolicy default-deny-all -n default
Then test from a throwaway pod with kubectl exec and a tool such as wget, confirming that the allowed path connects and every other destination times out.
Limits and alternatives
- Not a replacement for patching. These controls reduce what a compromised process can do. They do not stop the initial exploit. The vulnerable Log4j version still has to be upgraded.
- Kernel and runtime bugs remain. A non-root user and dropped capabilities make many escapes harder but not impossible. Keep the container runtime and kernel updated, since fixes such as the one for CVE-2019-5736 ship there.
- Read-only does not cover memory. Code can still run in memory, and anything written to a tmpfs or volume stays writable. Mount tmpfs with
noexecwhere the application allows it. - Policies need enforcement. A NetworkPolicy object does nothing on a cluster whose network plugin ignores it. Check that the plugin supports policy before relying on it.
- Capability tuning costs effort. Some images only start with extra capabilities, and overly broad egress allow rules weaken the default-deny.
- Cluster-wide enforcement. PodSecurityPolicy is deprecated as of Kubernetes 1.21. The built-in Pod Security admission arrives as a beta in 1.23, and third-party admission controllers can enforce the same rules until then.
- Stronger isolation. For untrusted or multi-tenant workloads, use rootless mode or user namespaces, a sandboxed runtime, or separate virtual machines. A hardware-virtualised VM gives each trust zone its own kernel, which containers on one node do not have.