Back to all articles

Why Does Docker Run in Layers?

A casual breakdown of why Docker executes in layers, how Copy-on-Write works under the hood, and the container filesystem lifecycle.

• Yahya
Docker Containers DevOps Linux

If you have ever spun up a container on a fresh machine, you have probably stared at a terminal output that looks something like this:

$ docker run -d -p 80:80 nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
7a5d12f3b9e4: Pull complete
9b3a721c5d80: Pull complete
5f410298a12e: Pull complete
d29a50e1837b: Pull complete
Digest: sha256:4d6423984534a742876...
Status: Downloaded newer image for nginx:latest

Instead of downloading a single neat installer like nginx.iso or nginx.zip, Docker pulls a cascade of distinct cryptographic hashes, each extracting its own slice of files in parallel.

Why does Docker do this? Why not just hand you one monolithic virtual disk and call it a day?

The answer lies in layered filesystems and how the container lifecycle manages changes from build time to runtime.

What Actually Is a Docker Layer?

Think of a Docker image not as a heavy virtual hard drive, but as a stack of semi-transparent tracing paper sheets.

Each layer represents an immutable snapshot of changes (added files, modified configs, or deleted binaries) introduced by a specific instruction in your Dockerfile:

FROM python:3.11-slim           # Layer 1: Base Debian rootfs + Python binary
WORKDIR /app                   # Layer 2: Working directory metadata
COPY requirements.txt .        # Layer 3: Dependency manifest
RUN pip install -r requirements.txt  # Layer 4: Installed packages
COPY . .                       # Layer 5: Your actual application code

When Docker builds this image:

  1. It executes FROM python:3.11-slim, pulling the pre-built base OS layer.
  2. It executes WORKDIR /app, noting that directory state as an incremental layer.
  3. It copies requirements.txt as another layer.
  4. It runs pip install, creating a layer containing just the downloaded libraries.
  5. It copies your source code as the final layer.

Every layer is stored as a compressed tarball identified by its SHA-256 hash. Once created, an image layer is 100% read-only and immutable. It never mutates.

The Glue: Union Filesystems and OverlayFS

How does Linux turn four or five stacked folders into what looks like a standard unified Linux root filesystem (/) inside your container?

Docker uses a storage driver mechanism known as a Union File System (most commonly OverlayFS):

  • Lower Directories (lowerdir): The stack of read-only image layers.
  • Merged Directory (mergeddir): The unified virtual view presented to your running processes.

If Layer 1 contains /etc/nginx/nginx.conf and Layer 4 replaces /etc/nginx/nginx.conf with a custom configuration, OverlayFS merges them such that the file in the higher layer hides the file in the lower layer. To your process running inside the container, there is only one /etc/nginx/nginx.conf.

The Container Lifecycle: Where Does My Data Go?

When you type docker run, Docker does not make a full copy of the 500 MB image. If it did, spinning up ten containers would take minutes and consume gigabytes of disk space.

Instead, Docker goes through a four-stage filesystem lifecycle:

[ Container Process View: /mergeddir ]
                 ▲
                 │
┌────────────────┴────────────────┐
│  Thin Read-Write Container Layer │  ◄── Added on "docker run" (ephemeral)
├─────────────────────────────────┤
│  Layer 5: Application Source    │  (read-only)
├─────────────────────────────────┤
│  Layer 4: Installed Packages    │  (read-only)
├─────────────────────────────────┤
│  Layer 3: Dependency Manifest   │  (read-only)
├─────────────────────────────────┤
│  Layer 1: Base Linux OS         │  (read-only)
└─────────────────────────────────┘

1. Ignition: Slapping on the Writable Layer

When the container starts, Docker mounts the entire stack of image layers as read-only, and attaches one very thin read-write layer (often called the container layer or upperdir) directly on top.

2. Copy-on-Write (CoW) in Action

  • Reading files: If your web server reads static assets, it reads directly down through the read-only image layers with zero overhead.
  • Modifying existing files: If a process writes to an existing file that belongs to an image layer, the storage driver intercepts the write, makes a duplicate of that file up into the writable container layer, and modifies the copy there. The original file in the lower image layer remains untouched.
  • Deleting files: If you run rm /app/config.json, Docker does not touch the read-only layer below. It creates a special whiteout character device in the writable layer that masks the lower file from view.

3. Container Stop (docker stop)

When the container stops, its read-write layer stays frozen on disk. You can inspect logs, check modified files, or start it back up (docker start) and pick up exactly where you left off.

4. Container Destruction (docker rm)

The moment you delete the container with docker rm, Docker simply unlinks and deletes that single, thin read-write layer. The underlying image layers stay untouched in the local cache, ready to power twenty other containers instantly.

Why This Architecture is a Game-Changer

  1. Storage Deduplication: If you run ten instances of an API container based on python:3.11-slim, that 150 MB base layer exists on your host disk exactly once. Ten running containers consume 150 MB total, plus a few megabytes of writable scratch space.
  2. Lightning-Fast Builds via Layer Caching: If you edit one line of code in app.py, Docker realizes that layers 1 through 4 have not changed. It fetches them from cache in milliseconds and only rebuilds layer 5.
  3. Network Bandwidth Savings: When you push version 1.2.0 of your microservice to an AWS ECR or Docker Hub registry, your CI/CD runner only uploads the 3 MB layer containing your new code, completely skipping the unchanged 300 MB operating system and library layers.

Key Takeaways for Production Dockerfiles

  • Order from least-frequent to most-frequent changes: Place OS package installs (apt-get) and dependency resolutions (npm install, pip install) above your application source code (COPY . .). That way, editing application code never invalidates your dependency cache.
  • Chain commands to keep layers lean: Each RUN creates a layer. Cleaning up package caches in a subsequent RUN step does not reduce the image size because the previous layer is already committed. Always run cleanup in the same step: apt-get update && apt-get install -y nginx && rm -rf /var/lib/apt/lists/*.
  • Use Multi-Stage Builds: Compile your binaries in a bloated build-stage layer, and COPY only the final stripped executable into a clean, minimal scratch or distroless runtime layer.

Further Reading

Back to articles