Docker Build Caching: Cutting Build Times by 80%

Your CI pipeline runs for 12 minutes. Eight of those minutes are Docker building the same dependencies you built yesterday. And the day before. And last week.

Docker’s layer caching should prevent this, but it doesn’t—because your Dockerfile invalidates the cache on line 3 when you copy the entire source directory. Every build starts from scratch, re-downloading packages and recompiling code that hasn’t changed.

This isn’t about tolerating slow builds. It’s about understanding how layer caching works and structuring Dockerfiles to exploit it.

How Docker Layer Caching Works

Docker builds images in layers. Each instruction in your Dockerfile creates a layer. When you rebuild, Docker checks if anything affecting that layer has changed. If not, it reuses the cached layer.

The cache invalidation is simple: if a layer changes, all subsequent layers are invalidated. This is where most Dockerfiles fail.

Bad Dockerfile that breaks caching:

FROM node:18
WORKDIR /app
COPY . .                    # Copies everything - invalidates on any file change
RUN npm install             # Runs every time, even if package.json unchanged
RUN npm run build

Any file change (source code, README, config) invalidates the cache at COPY . ., forcing npm install and npm run build to run again.

Better Dockerfile with cache-friendly layering:

FROM node:18
WORKDIR /app
COPY package.json package-lock.json ./    # Only copies dependency files
RUN npm install                            # Cached unless dependencies change
COPY . .                                   # Source changes don't invalidate npm install
RUN npm run build

Now npm install only runs when dependencies actually change. Source code changes skip straight to the copy and build steps.

Multi-Stage Builds for Smaller Images and Better Caching

Multi-stage builds separate build-time dependencies from runtime dependencies. This improves caching and reduces final image size.

Example: Go application with multi-stage build

# Build stage
FROM golang:1.21 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download              # Cached unless dependencies change
COPY . .
RUN CGO_ENABLED=0 go build -o app

# Runtime stage
FROM alpine:3.18
RUN apk add --no-cache ca-certificates
COPY --from=builder /app/app /app
ENTRYPOINT ["/app"]

The build stage runs in a full Go environment with all build tools. The runtime stage copies only the compiled binary into a minimal Alpine image. Build dependencies aren’t included in the final image.

The cache benefit: changing source code doesn’t invalidate go mod download. That layer is reused as long as go.mod and go.sum are unchanged.

BuildKit: Docker’s Modern Build Engine

Docker’s legacy builder has limitations. BuildKit is the newer build engine with better performance and more features.

Enable BuildKit:

export DOCKER_BUILDKIT=1
docker build .

Or make it permanent in /etc/docker/daemon.json:

{
  "features": {
    "buildkit": true
  }
}

BuildKit improvements:

• Parallel layer builds (independent stages build concurrently)
• Better cache management
• Cache mounts for package managers
• Secrets handling without leaking into layers

Cache Mounts: The BuildKit Secret Weapon

Cache mounts persist directories across builds without adding them to the image. This is huge for package managers.

Before cache mounts:

RUN npm install

This downloads packages from npm registry every time package.json changes. Slow, wasteful, hits rate limits.

With cache mounts:

RUN --mount=type=cache,target=/root/.npm \
    npm install

The /root/.npm directory (npm’s cache) persists between builds. npm checks its cache before downloading. Identical packages download once.

More cache mount examples:

# Go modules
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

# pip packages
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# apt packages
RUN --mount=type=cache,target=/var/cache/apt \
    apt-get update && apt-get install -y build-essential

Layer Ordering: Put Stable Layers First

Order Dockerfile instructions from least frequently changed to most frequently changed.

Suboptimal ordering:

FROM python:3.11
COPY . /app                          # Changes frequently
RUN pip install -r requirements.txt  # Runs on every code change

Optimized ordering:

FROM python:3.11
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt   # Cached unless requirements change
COPY . /app                                 # Source changes don't bust pip cache

Principle: Dependencies change less often than source code. Install dependencies before copying source.

.dockerignore: Stop Breaking the Cache

A .dockerignore file tells Docker what to exclude from the build context. Without it, Docker sends your entire directory (node_modules, .git, build artifacts) to the daemon.

Worse, changing ignored files invalidates COPY layers even though those files aren’t actually needed.

Create .dockerignore:

.git
.gitignore
node_modules
npm-debug.log
README.md
.env
.DS_Store
dist
build
*.log

This prevents:

• Sending unnecessary files to Docker daemon (faster context upload)
• Cache invalidation from irrelevant file changes
• Accidentally including secrets or large files

CI/CD-Specific Caching Strategies

CI environments are ephemeral. Each build runs in a clean environment with no local cache. You need external cache storage.

GitHub Actions: Docker Layer Caching

Using docker/build-push-action with cache:

- name: Build and push
  uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: myapp:latest
    cache-from: type=registry,ref=myapp:cache
    cache-to: type=registry,ref=myapp:cache,mode=max

This pushes build cache to a registry image. Subsequent builds pull the cache, reusing layers.

Mode options:

• mode=min: Only exports layers from final stage (smaller cache, less reuse)
• mode=max: Exports all layers from all stages (larger cache, maximum reuse)

GitLab CI: Docker-in-Docker Caching

build:
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_BUILDKIT: 1
  script:
    - docker pull $CI_REGISTRY_IMAGE:cache || true
    - docker build
        --cache-from $CI_REGISTRY_IMAGE:cache
        --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE:cache .
    - docker push $CI_REGISTRY_IMAGE:cache
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

Pull previous cache image, build with --cache-from, push updated cache.

AWS CodeBuild: Local Caching

CodeBuild supports local Docker layer caching:

version: 0.2
phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  build:
    commands:
      - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
artifacts:
  files:
    - '**/*'
cache:
  paths:
    - '/var/lib/docker/**/*'

The cache section preserves /var/lib/docker between builds, keeping layers locally.

Measuring Cache Effectiveness

Track cache hit rate to verify improvements.

Check build output:

=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 500B
=> [internal] load .dockerignore
=> => transferring context: 100B
=> [internal] load metadata for docker.io/library/node:18
=> [1/4] FROM docker.io/library/node:18
=> CACHED [2/4] COPY package.json package-lock.json ./
=> CACHED [3/4] RUN npm install
=> [4/4] COPY . .

“CACHED” means the layer was reused. Count cached vs. rebuilt layers.

Time builds:

time docker build -t myapp .

Before optimization: 600 seconds After optimization: 45 seconds

That’s a 92% reduction—typical for projects that weren’t exploiting caching.

Common Caching Mistakes

1. Copying source before installing dependencies:

COPY . .
RUN npm install

Every source change reinstalls dependencies.

2. Combining unrelated commands:

RUN apt-get update && apt-get install -y curl git vim && npm install

Changing npm dependencies invalidates the entire apt installation.

3. Using ADD instead of COPY:

ADD has magic behavior (auto-extraction, URL downloads) that breaks caching. Use COPY unless you specifically need ADD.

4. Not using .dockerignore:

Cache invalidates on changes to files you don’t even use in the build.

5. Running apt-get update without install:

RUN apt-get update
RUN apt-get install -y curl

The update layer caches indefinitely, using stale package indexes. Combine them:

RUN apt-get update && apt-get install -y curl

Advanced: Dependency Caching Patterns

Python with requirements.txt variants:

FROM python:3.11
WORKDIR /app

# Base dependencies (rarely change)
COPY requirements-base.txt .
RUN pip install -r requirements-base.txt

# Development dependencies (change occasionally)
COPY requirements-dev.txt .
RUN pip install -r requirements-dev.txt

# Application code
COPY . .

Splitting requirements into tiers means base dependencies cache even when dev dependencies change.

Node.js with workspace monorepos:

FROM node:18
WORKDIR /app

# Copy all package.json files first
COPY package.json package-lock.json ./
COPY packages/*/package.json ./packages/

# Install all dependencies
RUN npm install

# Copy source
COPY . .
RUN npm run build

Changes to one package’s source don’t invalidate root dependency installation.

The Practical Impact

A real example from a client project:

Before optimization:

• Build time: 11 minutes
• Cache hit rate: ~20%
• npm install ran on every build
• Full apt-get update/install every time

After optimization:

• Build time: 90 seconds
• Cache hit rate: ~85%
• Dependencies only reinstalled when changed
• Base image layers cached

Changes made:

1. Reordered Dockerfile to copy dependencies before source
2. Added .dockerignore to exclude node_modules, .git
3. Enabled BuildKit with cache mounts for npm
4. Split multi-stage build (build stage vs. runtime stage)
5. Configured GitHub Actions to push/pull cache from registry

Total time investment: ~3 hours Time saved per build: ~9.5 minutes Builds per day: ~40 Monthly time savings: ~250 hours of CI runner time

At $0.008/minute for GitHub Actions, that’s $200/month saved in CI costs. More importantly, developers get feedback 10x faster.

Start Here

If you’re new to Docker build optimization:

1. Add .dockerignore - Immediate win, zero downside
2. Reorder your Dockerfile - Copy dependency files before source
3. Enable BuildKit - Set DOCKER_BUILDKIT=1
4. Add cache mounts - Use --mount=type=cache for package managers
5. Measure - Time builds before and after

Don’t optimize prematurely, but don’t ignore free performance either. A well-cached Dockerfile is just as readable as a poorly cached one—it’s about knowing the rules.