Running Kubernetes e2e_node tests locally with Lima VM

README.md

Running Kubernetes e2e_node Tests Locally with Lima

This guide documents how to run Kubernetes single-node e2e tests (e2e_node) locally on macOS using Lima. Lima creates lightweight Linux VMs that can run the full kubelet with systemd, containerd, and all dependencies required for e2e_node tests.

Why Lima?

• Full Linux environment: e2e_node tests require systemd, cgroups v2, and a real container runtime
• Docker Desktop limitations: Nested containers don't work properly on Docker Desktop for Mac (fails with "failed to mount rootfs component: invalid argument")
• Fast iteration: Mount your local Kubernetes source code directly into the VM
• Architecture support: Works on both Intel and Apple Silicon Macs

Prerequisites

Install Lima

# Using Homebrew
brew install lima

# Verify installation
limactl --version

Clone Kubernetes Source

mkdir -p ~/go/src/k8s.io
cd ~/go/src/k8s.io
git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes

Quick Start

1. Create the Lima VM

Important: Before starting, edit k8s-e2e-node.yaml to update the mount paths to match your local setup.

# Start the VM (first time takes ~2-3 minutes to download image and provision)
limactl start k8s-e2e-node.yaml

# Check VM status
limactl list

2. Enter the VM

limactl shell k8s-e2e-node

3. Run Tests

Inside the VM:

# Navigate to Kubernetes source
cd ~/go/src/k8s.io/kubernetes

# Run a specific test
sudo -E make test-e2e-node \
  FOCUS="your test pattern here" \
  PARALLELISM=1 \
  KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

Important: Running Serial/Disruptive Tests

The default make test-e2e-node command skips tests tagged with [Serial], [Slow], or [Flaky].

If your test has any of these tags (check the test name), you must run ginkgo directly:

# First, build the test binaries (run make once)
sudo -E make test-e2e-node FOCUS="." PARALLELISM=1 KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

# Then run ginkgo directly without the skip flags
sudo -E _output/local/go/bin/ginkgo \
  -timeout=24h \
  -focus="Does not keep device plugin assignments across node reboots" \
  -skip="\[Flaky\]" \
  _output/local/go/bin/e2e_node.test \
  -- \
  --v 4 \
  --report-dir=/tmp/_artifacts \
  --node-name $(hostname) \
  --kubelet-flags="--cluster-domain=cluster.local" \
  --dns-domain="cluster.local" \
  --prepull-images=false \
  --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
  --kubelet-config-file="/etc/kubelet-config.yaml"

Why This Matters

The Makefile runs ginkgo with these skip patterns:

-skip="\[Flaky\]|\[Slow\]|\[Serial\]"

So if you're running a test like:

[sig-node] Device Plugin [Serial] [Feature:DevicePlugin] DevicePlugin [Serial] [Disruptive] Does not keep device plugin assignments...

It will be silently skipped and show "0 of 1075 Specs" ran!

Configuration Details

VM Resources

The default configuration allocates:

• CPUs: 4
• Memory: 8 GiB
• Disk: 50 GiB

Adjust these in k8s-e2e-node.yaml based on your needs. Some tests may require more memory.

Installed Components

The provisioning script installs:

Component	Version	Purpose
Go	1.25.5	Build Kubernetes (update to match go.mod)
containerd	(Lima default)	Container runtime
runc	1.2.4	OCI runtime
CNI plugins	1.6.2	Container networking

Key Configurations

1. containerd: Configured with SystemdCgroup = true for cgroups v2 compatibility
2. kubelet: Pre-configured with systemd cgroup driver at /etc/kubelet-config.yaml
3. CNI: Bridge networking configured in /etc/cni/net.d/

Common Test Patterns

Run a Single Non-Serial Test

sudo -E make test-e2e-node \
  FOCUS="exact test name here" \
  PARALLELISM=1 \
  KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

Run Tests Matching a Pattern

# All device plugin tests (non-serial ones)
sudo -E make test-e2e-node \
  FOCUS="DevicePlugin" \
  PARALLELISM=1 \
  KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

Run Serial Tests (using ginkgo directly)

# After building once with make, run directly:
sudo -E _output/local/go/bin/ginkgo \
  -timeout=24h \
  -focus="YourSerialTestPattern" \
  -skip="\[Flaky\]" \
  _output/local/go/bin/e2e_node.test \
  -- \
  --v 4 \
  --report-dir=/tmp/_artifacts \
  --node-name $(hostname) \
  --kubelet-flags="--cluster-domain=cluster.local" \
  --dns-domain="cluster.local" \
  --prepull-images=false \
  --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
  --kubelet-config-file="/etc/kubelet-config.yaml"

Run with Verbose Output

sudo -E make test-e2e-node \
  FOCUS="your test" \
  PARALLELISM=1 \
  KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml \
  2>&1 | tee /tmp/test-output.log

Run a Test Multiple Times (Flake Detection)

for i in {1..10}; do
  echo "=== Run $i ==="
  sudo -E _output/local/go/bin/ginkgo \
    -timeout=24h \
    -focus="Your Test Pattern" \
    -skip="\[Flaky\]" \
    _output/local/go/bin/e2e_node.test \
    -- \
    --kubelet-config-file="/etc/kubelet-config.yaml" \
    --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
    2>&1 | tee /tmp/run_$i.log

  if ! grep -q "1 Passed" /tmp/run_$i.log; then
    echo "FAILED on run $i"
    break
  fi
  echo "Run $i passed"
done

VM Management

Stop the VM

limactl stop k8s-e2e-node

Start a Stopped VM

limactl start k8s-e2e-node

Delete the VM

limactl delete k8s-e2e-node

SSH into the VM

limactl shell k8s-e2e-node

# Or use SSH directly
ssh -F ~/.lima/k8s-e2e-node/ssh.config lima-k8s-e2e-node

Copy Files

# Copy from host to VM
limactl copy myfile.txt k8s-e2e-node:/tmp/

# Copy from VM to host
limactl copy k8s-e2e-node:/tmp/results.log ./

Troubleshooting

Test Shows "0 of N Specs" Ran

Your test is probably tagged with [Serial] and being skipped. Run ginkgo directly without the -skip="\[Serial\]" flag. See "Running Serial/Disruptive Tests" section above.

Test Fails to Start Kubelet

Check containerd is running:

sudo systemctl status containerd

Check kubelet logs if it started:

sudo journalctl -u kubelet -f

Port Already in Use

If you see "port 6443 already in use", a previous test run didn't clean up:

sudo lsof -i :6443
sudo kill <PID>

CNI Errors

Verify CNI plugins are installed:

ls -la /opt/cni/bin/

Check CNI configuration:

cat /etc/cni/net.d/10-containerd-net.conflist

Cgroup Errors

Verify cgroups v2 is in use:

mount | grep cgroup

Check containerd config has SystemdCgroup enabled:

grep SystemdCgroup /etc/containerd/config.toml

View Test Artifacts

Test artifacts are written to /tmp/_artifacts/:

ls -la /tmp/_artifacts/

Customizing the Configuration

Change Mounted Directories

Edit the mounts section in k8s-e2e-node.yaml:

mounts:
  - location: "/path/to/your/kubernetes"
    mountPoint: "/home/YOUR_USER.linux/go/src/k8s.io/kubernetes"
    writable: true

Note: The mountPoint path inside the VM should match where you'll cd to run tests.

Use a Different Go Version

Check the go.mod file in your Kubernetes source for the required version, then edit the provisioning script:

provision:
  - mode: system
    script: |
      GO_VERSION=1.23.0  # Change this to match go.mod
      ...

Add Additional Packages

Add to the apt-get install list in the provisioning script.

Example: Investigating a Flaky Device Plugin Test

This example shows how to run the device plugin assignment test that was being investigated:

# Enter the VM
limactl shell k8s-e2e-node

# Navigate to source
cd ~/go/src/k8s.io/kubernetes

# Build test binaries first
sudo -E make test-e2e-node FOCUS="." PARALLELISM=1 KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml 2>&1 | head -50

# Run the Serial test directly with ginkgo (skipping only Flaky, not Serial)
sudo -E _output/local/go/bin/ginkgo \
  -timeout=24h \
  -focus="Does not keep device plugin assignments across node reboots" \
  -skip="\[Flaky\]" \
  _output/local/go/bin/e2e_node.test \
  -- \
  --v 4 \
  --report-dir=/tmp/_artifacts/deviceplugin \
  --node-name $(hostname) \
  --kubelet-flags="--cluster-domain=cluster.local" \
  --dns-domain="cluster.local" \
  --prepull-images=false \
  --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
  --kubelet-config-file="/etc/kubelet-config.yaml"

Running Multiple Iterations

for i in {1..100}; do
  echo "=== Run $i at $(date) ==="
  sudo -E _output/local/go/bin/ginkgo \
    -timeout=24h \
    -focus="Does not keep device plugin assignments across node reboots" \
    -skip="\[Flaky\]" \
    _output/local/go/bin/e2e_node.test \
    -- \
    --report-dir=/tmp/_artifacts/run$i \
    --node-name $(hostname) \
    --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
    --kubelet-config-file="/etc/kubelet-config.yaml" \
    2>&1 | tee /tmp/run_$i.log

  if ! grep -q "1 Passed" /tmp/run_$i.log; then
    echo "FLAKE DETECTED on run $i"
    break
  fi
  echo "Run $i passed"
done

Notes

• The VM uses your host's Kubernetes source via mounts, so any changes you make on the host are immediately visible in the VM
• Test artifacts are stored in /tmp/_artifacts/ inside the VM
• The kubelet config at /etc/kubelet-config.yaml uses systemd cgroup driver which is required for cgroups v2
• Important: make test-e2e-node skips [Serial] tests by default - use ginkgo directly for those
• Each test run takes approximately 30-40 seconds
• Flaky tests in CI may be stable locally due to consistent timing in dedicated VM (vs shared GCE infrastructure)

k8s-e2e-node.yaml

# Lima VM configuration for Kubernetes e2e_node tests
#
# Usage:
#   1. Edit the 'mounts' section to point to your Kubernetes source
#   2. Run: limactl start k8s-e2e-node.yaml
#   3. Enter VM: limactl shell k8s-e2e-node
#   4. Run tests: cd ~/go/src/k8s.io/kubernetes && sudo -E make test-e2e-node FOCUS="..." PARALLELISM=1 KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml
#
# Requirements:
#   - Lima (brew install lima)
#   - Kubernetes source cloned locally

# VM resources - adjust based on your machine and test requirements
cpus: 4
memory: "8GiB"
disk: "50GiB"

# Ubuntu 24.04 LTS - supports both Intel and Apple Silicon
images:
  - location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
    arch: "aarch64"
  - location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
    arch: "x86_64"

# Mount your Kubernetes source code
# IMPORTANT: Update these paths to match your local setup
mounts:
  # Mount Kubernetes source - UPDATE THIS PATH
  - location: "~/go/src/k8s.io/kubernetes"
    mountPoint: "/home/{{.User}}.linux/go/src/k8s.io/kubernetes"
    writable: true
  # Optional: Mount home directory for access to other files
  - location: "~"
    writable: true

# Enable containerd (Lima provides this)
containerd:
  system: true
  user: false

# Provisioning script - runs on first boot
provision:
  - mode: system
    script: |
      #!/bin/bash
      set -eux -o pipefail

      echo "=== Installing base dependencies ==="
      apt-get update
      apt-get install -y \
        build-essential \
        pkg-config \
        libseccomp-dev \
        libseccomp2 \
        btrfs-progs \
        libbtrfs-dev \
        iptables \
        iproute2 \
        iputils-ping \
        procps \
        ethtool \
        socat \
        conntrack \
        ebtables \
        kmod \
        util-linux \
        ca-certificates \
        gnupg \
        jq \
        rsync \
        psmisc \
        curl \
        wget \
        git

      echo "=== Installing Go ==="
      # Update this version as needed - check go.mod in kubernetes repo
      GO_VERSION=1.25.5
      GOARCH=$(dpkg --print-architecture)
      curl -sSL "https://go.dev/dl/go${GO_VERSION}.linux-${GOARCH}.tar.gz" | tar -C /usr/local -xzf -

      # Set up Go environment for all users
      cat > /etc/profile.d/golang.sh << 'GOEOF'
      export PATH="/usr/local/go/bin:$HOME/go/bin:$PATH"
      export GOPATH="$HOME/go"
      GOEOF

      echo "=== Installing runc ==="
      RUNC_VERSION=1.2.4
      RUNC_ARCH=$(dpkg --print-architecture)
      curl -sSL -o /usr/local/sbin/runc \
        "https://github.com/opencontainers/runc/releases/download/v${RUNC_VERSION}/runc.${RUNC_ARCH}"
      chmod +x /usr/local/sbin/runc

      echo "=== Installing CNI plugins ==="
      CNI_VERSION=1.6.2
      CNI_ARCH=$(dpkg --print-architecture)
      mkdir -p /opt/cni/bin
      curl -sSL \
        "https://github.com/containernetworking/plugins/releases/download/v${CNI_VERSION}/cni-plugins-linux-${CNI_ARCH}-v${CNI_VERSION}.tgz" \
        | tar -C /opt/cni/bin -xzf -

      echo "=== Configuring CNI ==="
      mkdir -p /etc/cni/net.d
      cat > /etc/cni/net.d/10-containerd-net.conflist << 'CNIEOF'
      {
        "cniVersion": "1.0.0",
        "name": "containerd-net",
        "plugins": [
          {
            "type": "bridge",
            "bridge": "cni0",
            "isGateway": true,
            "ipMasq": true,
            "promiscMode": true,
            "ipam": {
              "type": "host-local",
              "ranges": [
                [{"subnet": "10.88.0.0/16"}]
              ],
              "routes": [
                { "dst": "0.0.0.0/0" }
              ]
            }
          },
          {
            "type": "portmap",
            "capabilities": {"portMappings": true}
          }
        ]
      }
      CNIEOF

      echo "=== Configuring containerd with SystemdCgroup ==="
      mkdir -p /etc/containerd
      containerd config default > /etc/containerd/config.toml
      # Enable SystemdCgroup for cgroups v2 compatibility
      sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml
      systemctl restart containerd

      echo "=== Enabling systemd resource accounting ==="
      mkdir -p /etc/systemd/system.conf.d
      cat > /etc/systemd/system.conf.d/kubernetes-accounting.conf << 'SYSEOF'
      [Manager]
      DefaultCPUAccounting=yes
      DefaultMemoryAccounting=yes
      SYSEOF

      echo "=== Creating kubelet directories ==="
      mkdir -p /var/lib/kubelet /var/log/pods /var/log/containers /tmp/_artifacts

      echo "=== Creating kubelet config ==="
      cat > /etc/kubelet-config.yaml << 'KUBELETEOF'
      apiVersion: kubelet.config.k8s.io/v1beta1
      kind: KubeletConfiguration
      # Use systemd cgroup driver (required for cgroups v2)
      cgroupDriver: systemd
      cgroupRoot: /
      # Allow running with swap (Lima VMs may have swap)
      failSwapOn: false
      # Pod networking
      podCIDR: "10.100.0.0/24"
      # Faster checks for testing
      volumeStatsAggPeriod: 10s
      fileCheckFrequency: 10s
      # Eviction settings
      evictionPressureTransitionPeriod: 30s
      evictionHard:
        memory.available: 250Mi
        nodefs.available: 10%
        nodefs.inodesFree: 5%
      evictionMinimumReclaim:
        nodefs.available: 5%
        nodefs.inodesFree: 5%
      # Don't serialize image pulls
      serializeImagePulls: false
      KUBELETEOF

      # Disable swap (optional since we set failSwapOn: false)
      swapoff -a || true

      echo "=== Provisioning complete! ==="

# Message shown after VM starts
message: |
  ============================================================
  Kubernetes e2e_node test VM is ready!
  ============================================================

  To run tests:
    limactl shell k8s-e2e-node

  Inside the VM:
    cd ~/go/src/k8s.io/kubernetes

    # Run a specific test
    sudo -E make test-e2e-node \
      FOCUS="YourTestPattern" \
      PARALLELISM=1 \
      KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

  Examples:
    # Device plugin tests
    sudo -E make test-e2e-node FOCUS="DevicePlugin" PARALLELISM=1 KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml

    # Run directly with ginkgo (after first build)
    sudo -E _output/local/go/bin/ginkgo -focus="YourTest" _output/local/go/bin/e2e_node.test -- \
      --kubelet-config-file="/etc/kubelet-config.yaml" \
      --container-runtime-endpoint=unix:///run/containerd/containerd.sock

  Useful commands:
    - Check containerd: sudo systemctl status containerd
    - View kubelet logs: sudo journalctl -u kubelet -f
    - Test artifacts: ls /tmp/_artifacts/

kubelet-config.yaml

# Kubelet configuration for e2e_node tests
# This file is created by the Lima provisioning script at /etc/kubelet-config.yaml
#
# Key settings:
#   - cgroupDriver: systemd (required for cgroups v2)
#   - failSwapOn: false (Lima VMs may have swap enabled)
#   - Fast check intervals for quicker test execution

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

# Cgroup configuration
# Use systemd cgroup driver - required for cgroups v2 (modern Linux)
cgroupDriver: systemd
cgroupRoot: /

# Allow running with swap enabled
# Lima VMs may have swap, and we don't want tests to fail for that reason
failSwapOn: false

# Pod networking
# This CIDR is used when no node controller is present (standalone e2e_node tests)
podCIDR: "10.100.0.0/24"

# Faster check intervals for testing
# These reduce the time tests need to wait for kubelet to notice changes
volumeStatsAggPeriod: 10s
fileCheckFrequency: 10s

# Eviction configuration
# Slightly relaxed settings for testing in VMs with limited resources
evictionPressureTransitionPeriod: 30s
evictionHard:
  memory.available: 250Mi
  nodefs.available: 10%
  nodefs.inodesFree: 5%
evictionMinimumReclaim:
  nodefs.available: 5%
  nodefs.inodesFree: 5%

# Image pulling
# Don't serialize - allows parallel image pulls for faster test setup
serializeImagePulls: false

run-e2e-test.sh

#!/bin/bash
#
# Helper script to run Kubernetes e2e_node tests in Lima VM
#
# Usage:
#   ./run-e2e-test.sh "TestPattern"
#   ./run-e2e-test.sh "DevicePlugin" 5        # Run 5 times
#   ./run-e2e-test.sh "Eviction" 1 "--skip=Slow"
#
# This script:
#   1. Enters the Lima VM
#   2. Runs the specified test pattern
#   3. Optionally runs multiple iterations to detect flaky tests

set -e

# Configuration
VM_NAME="${LIMA_VM_NAME:-k8s-e2e-node}"
K8S_PATH="${K8S_PATH:-~/go/src/k8s.io/kubernetes}"
KUBELET_CONFIG="/etc/kubelet-config.yaml"

# Arguments
TEST_PATTERN="${1:-}"
ITERATIONS="${2:-1}"
EXTRA_ARGS="${3:-}"

if [ -z "$TEST_PATTERN" ]; then
    echo "Usage: $0 <test-pattern> [iterations] [extra-args]"
    echo ""
    echo "Examples:"
    echo "  $0 'DevicePlugin'                    # Run DevicePlugin tests once"
    echo "  $0 'Eviction' 10                     # Run Eviction tests 10 times"
    echo "  $0 'Pod' 1 '--skip=Slow'             # Run Pod tests, skip slow ones"
    echo ""
    echo "Environment variables:"
    echo "  LIMA_VM_NAME  - Lima VM name (default: k8s-e2e-node)"
    echo "  K8S_PATH      - Path to Kubernetes source in VM"
    exit 1
fi

echo "=========================================="
echo "Running e2e_node tests in Lima VM"
echo "=========================================="
echo "VM:         $VM_NAME"
echo "Pattern:    $TEST_PATTERN"
echo "Iterations: $ITERATIONS"
echo "Extra args: $EXTRA_ARGS"
echo "=========================================="

# Check if VM is running
if ! limactl list | grep -q "$VM_NAME.*Running"; then
    echo "Starting Lima VM: $VM_NAME"
    limactl start "$VM_NAME"
fi

# Build the test command
TEST_CMD="cd $K8S_PATH && sudo -E make test-e2e-node FOCUS=\"$TEST_PATTERN\" PARALLELISM=1 KUBELET_CONFIG_FILE=$KUBELET_CONFIG $EXTRA_ARGS"

if [ "$ITERATIONS" -eq 1 ]; then
    # Single run
    echo "Running test..."
    limactl shell "$VM_NAME" -- bash -c "$TEST_CMD"
else
    # Multiple iterations
    echo "Running $ITERATIONS iterations..."
    limactl shell "$VM_NAME" -- bash -c "
        cd $K8S_PATH
        passed=0
        failed=0
        for i in \$(seq 1 $ITERATIONS); do
            echo ''
            echo '=========================================='
            echo \"Run \$i of $ITERATIONS\"
            echo '=========================================='

            if sudo -E make test-e2e-node FOCUS=\"$TEST_PATTERN\" PARALLELISM=1 KUBELET_CONFIG_FILE=$KUBELET_CONFIG $EXTRA_ARGS 2>&1 | tee /tmp/run_\$i.log; then
                if grep -q '0 Failed' /tmp/run_\$i.log; then
                    ((passed++))
                    echo \"Run \$i: PASSED\"
                else
                    ((failed++))
                    echo \"Run \$i: FAILED\"
                    echo 'Stopping due to failure'
                    break
                fi
            else
                ((failed++))
                echo \"Run \$i: FAILED (non-zero exit)\"
                echo 'Stopping due to failure'
                break
            fi
        done

        echo ''
        echo '=========================================='
        echo 'Summary'
        echo '=========================================='
        echo \"Passed: \$passed\"
        echo \"Failed: \$failed\"
        echo \"Total:  \$((passed + failed))\"
    "
fi

echo ""
echo "Test run complete."

setup.sh

#!/bin/bash
#
# Quick setup script for Lima-based Kubernetes e2e_node testing
#
# This script:
#   1. Checks prerequisites
#   2. Updates the Lima config with your paths
#   3. Starts the VM
#
# Usage:
#   ./setup.sh /path/to/kubernetes/source

set -e

K8S_SOURCE="${1:-}"
VM_NAME="k8s-e2e-node"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CONFIG_FILE="$SCRIPT_DIR/k8s-e2e-node.yaml"

echo "=========================================="
echo "Kubernetes e2e_node Lima VM Setup"
echo "=========================================="

# Check prerequisites
echo "Checking prerequisites..."

if ! command -v limactl &> /dev/null; then
    echo "ERROR: limactl not found. Install with: brew install lima"
    exit 1
fi
echo "  limactl: $(limactl --version)"

if [ -z "$K8S_SOURCE" ]; then
    # Try to find kubernetes source
    if [ -d "$HOME/go/src/k8s.io/kubernetes" ]; then
        K8S_SOURCE="$HOME/go/src/k8s.io/kubernetes"
    else
        echo ""
        echo "ERROR: Kubernetes source path required"
        echo "Usage: $0 /path/to/kubernetes/source"
        echo ""
        echo "Example:"
        echo "  $0 ~/go/src/k8s.io/kubernetes"
        exit 1
    fi
fi

if [ ! -d "$K8S_SOURCE" ]; then
    echo "ERROR: Kubernetes source not found at: $K8S_SOURCE"
    exit 1
fi
echo "  Kubernetes source: $K8S_SOURCE"

if [ ! -f "$CONFIG_FILE" ]; then
    echo "ERROR: Lima config not found at: $CONFIG_FILE"
    exit 1
fi
echo "  Lima config: $CONFIG_FILE"

# Check if VM already exists
if limactl list 2>/dev/null | grep -q "$VM_NAME"; then
    echo ""
    echo "VM '$VM_NAME' already exists."
    echo ""
    read -p "Delete and recreate? (y/N) " -n 1 -r
    echo
    if [[ $REPLY =~ ^[Yy]$ ]]; then
        echo "Stopping and deleting existing VM..."
        limactl stop "$VM_NAME" 2>/dev/null || true
        limactl delete "$VM_NAME" 2>/dev/null || true
    else
        echo "Keeping existing VM. Use 'limactl shell $VM_NAME' to enter."
        exit 0
    fi
fi

# Create a temporary config with updated paths
echo ""
echo "Creating VM configuration..."
TEMP_CONFIG=$(mktemp)
trap "rm -f $TEMP_CONFIG" EXIT

# Update the mount path in the config
# Replace the placeholder path with the actual path
sed "s|~/go/src/k8s.io/kubernetes|$K8S_SOURCE|g" "$CONFIG_FILE" > "$TEMP_CONFIG"

echo "Starting Lima VM (this may take 2-3 minutes on first run)..."
echo ""
limactl start "$TEMP_CONFIG" --name "$VM_NAME"

echo ""
echo "=========================================="
echo "Setup complete!"
echo "=========================================="
echo ""
echo "To enter the VM:"
echo "  limactl shell $VM_NAME"
echo ""
echo "To run tests (inside VM):"
echo "  cd ~/go/src/k8s.io/kubernetes"
echo "  sudo -E make test-e2e-node FOCUS=\"YourTest\" PARALLELISM=1 KUBELET_CONFIG_FILE=/etc/kubelet-config.yaml"
echo ""
echo "To stop the VM:"
echo "  limactl stop $VM_NAME"
echo ""