Lima + containerd + nerdctl + BuildKit setup guide (macOS Apple Silicon)

CONTAINERS.md

Container Stack: Lima + containerd + nerdctl + BuildKit

Installation (clean macOS, Apple Silicon)

1. Install Lima

brew install lima

2. Create VM template

Save as ~/lima-default.yaml:

minimumLimaVersion: 2.0.0

base:
  - template:_images/ubuntu-24.04
  - template:_default/mounts

vmType: vz
cpus: 4
memory: 8GiB
disk: 40GiB
mountType: virtiofs
mountInotify: true

vmOpts:
  vz:
    rosetta:
      enabled: true
      binfmt: true

containerd:
  system: true
  user: true

hostResolver:
  enabled: true
  hosts:
    host.docker.internal: host.lima.internal

3. Create and start VM

limactl create --name=default ~/lima-default.yaml
limactl start default

4. Install QEMU binfmt (multi-arch beyond x86_64)

lima sudo nerdctl run --privileged --rm tonistiigi/binfmt --install all

5. Create nerdctl symlink

sudo ln -s /opt/homebrew/bin/nerdctl.lima /usr/local/bin/nerdctl

6. Add shell functions to ~/.bashrc (or ~/.zshrc)

# Container runtime: Lima + containerd + nerdctl + BuildKit
nerdctl() {
    /opt/homebrew/bin/limactl shell --preserve-env "${LIMA_INSTANCE:-default}" sudo nerdctl "$@"
}
export -f nerdctl

nerdctl-rootless() {
    /opt/homebrew/bin/limactl shell --preserve-env "${LIMA_INSTANCE:-default}" nerdctl "$@"
}
export -f nerdctl-rootless

docker() {
    /opt/homebrew/bin/limactl shell --preserve-env "${LIMA_INSTANCE:-default}" sudo nerdctl "$@"
}
export -f docker

docker-compose() {
    /opt/homebrew/bin/limactl shell --preserve-env "${LIMA_INSTANCE:-default}" sudo nerdctl compose "$@"
}
export -f docker-compose

7. VS Code (optional)

Install ms-azuretools.vscode-docker extension, then add to settings:

{
    "docker.dockerPath": "/usr/local/bin/nerdctl"
}

8. Verify

source ~/.bashrc
nerdctl run --rm alpine echo "it works"
docker run --rm --platform=amd64 alpine uname --machine   # x86_64

---

Architecture

┌─ macOS (Apple Silicon) ──────────────────────────────┐
│                                                       │
│  nerdctl()        → lima sudo nerdctl (rootful)       │
│  nerdctl-rootless → lima nerdctl (rootless)           │
│  docker()         → lima sudo nerdctl (compat alias)  │
│  docker-compose() → lima sudo nerdctl compose         │
│                                                       │
└───────────────────────┬───────────────────────────────┘
                        │ SSH (limactl shell)
┌───────────────────────▼───────────────────────────────┐
│  Lima VM "default"                                     │
│  Ubuntu 24.04 · vz · virtiofs · 4 CPU · 8 GB · 40 GB │
│                                                        │
│  containerd (system + user)                            │
│  nerdctl · BuildKit                                    │
│                                                        │
│  Multi-arch:                                           │
│    arm64   → native                                    │
│    x86_64  → Rosetta (near-native speed)               │
│    arm/v7, s390x, ppc64le → QEMU binfmt               │
│                                                        │
│  Mounts: ~/  (virtiofs, read-only by default)          │
│  DNS: host.docker.internal → host machine              │
└────────────────────────────────────────────────────────┘

Daily Usage

# Pull / run / stop / rm — same as docker
nerdctl pull alpine:latest
nerdctl run --detach --name myapp --publish 8080:80 nginx:latest
nerdctl stop myapp
nerdctl rm myapp

# List
nerdctl ps
nerdctl images

# Build (BuildKit)
nerdctl build --tag myimage:latest .

# Compose
nerdctl compose up --detach
nerdctl compose down

# docker alias works identically
docker run --rm alpine echo "works"
docker-compose up --detach

Multi-arch

# Default: native arm64
nerdctl run --rm alpine uname --machine
# aarch64

# Force amd64 (Rosetta — fast)
nerdctl run --rm --platform=amd64 alpine uname --machine
# x86_64

# Other architectures (QEMU — slower)
nerdctl run --rm --platform=linux/arm/v7 alpine uname --machine
# armv7l

# Multi-platform build
nerdctl build --platform=linux/amd64,linux/arm64 --tag myimage:latest .

How multi-arch works (single VM)

Container binary (any arch)
        ↓
Linux kernel (arm64) — binfmt_misc
        ↓ reads ELF header → determines architecture
        ↓
┌─ arm64   → runs natively
├─ x86_64  → Rosetta (VZ framework passthrough, near-native)
└─ arm/v7+ → QEMU user-mode emulation (slower)

QEMU binfmt registrations don't survive VM reboot. Re-register after limactl stop/start:

lima sudo nerdctl run --privileged --rm tonistiigi/binfmt --install all

Rosetta survives reboots — it's provided by the VM framework.

Rootful vs Rootless

# Rootful (default) — maximum compatibility
nerdctl run --rm alpine id
# uid=0(root)

# Rootless — better security, some limitations
nerdctl-rootless run --rm alpine id
# uid=0(root) inside user namespace

Rootful is the default because most images expect root. Use rootless for untrusted workloads.

VM Management

limactl list                  # status
limactl stop default          # stop VM
limactl start default         # start VM
lima                          # SSH into VM
lima sudo nerdctl version     # run command inside VM

VM config: ~/.lima/default/lima.yaml

Volume Mounts

Home directory (~/) is mounted via virtiofs (read-only by default):

nerdctl run --rm --volume "${HOME}/project:/app:ro" alpine ls /app

Paths outside ~/ (like /tmp) are NOT mounted. Use ~/ for build contexts and volume mounts.

Key Differences from Docker

Feature	Docker	nerdctl
CLI	docker	nerdctl (aliased as docker)
Runtime	Docker Engine (dockerd)	containerd
Build	BuildKit via dockerd	BuildKit standalone
Socket	/var/run/docker.sock	No socket (CLI only)
Compose	docker compose	nerdctl compose
Image format	OCI / Docker	OCI

File Locations

What	Where
VM config	~/.lima/default/lima.yaml
VM disk	~/.lima/default/
nerdctl symlink	/usr/local/bin/nerdctl → nerdctl.lima
Shell functions	~/.bashrc (bottom of file)
VS Code settings	docker.dockerPath = /usr/local/bin/nerdctl
Lima binary	/opt/homebrew/bin/limactl

Troubleshooting

# VM won't start
limactl list                          # check status
limactl stop default                  # force stop
limactl start default                 # restart

# containerd issues inside VM
lima sudo systemctl status containerd
lima sudo systemctl restart containerd

# Disk full
lima df --human-readable              # check disk usage
lima sudo nerdctl system prune --all  # clean images/containers

# binfmt lost after reboot
lima sudo nerdctl run --privileged --rm tonistiigi/binfmt --install all