k3s-installation-guide.md

k3s-installation-guide.md

K3s Kubernetes Cluster Installation Manual

Ubuntu 22.04 - Complete Guide

Version 1.0 | October 2025
Author: Cynthia Vázquez
Status: Production Ready

---

Table of Contents

1. Introduction
2. Prerequisites
3. Master Node Installation
4. Worker Nodes Installation
5. Troubleshooting
6. Firewall Configuration
7. SSH Configuration
8. File Transfer Between Nodes
9. Container Management
10. References
11. Appendix: Quick Command Reference

---

1. Introduction

This manual provides step-by-step instructions for installing and configuring a K3s Kubernetes cluster on Ubuntu 22.04 with one master node and two worker nodes.

Reference Guide

• How to set up a K3s Kubernetes Cluster on Ubuntu 22.04

---

2. Prerequisites

2.1 System Requirements

• 3 Ubuntu 22.04 servers
• Network connectivity between all nodes
• Root or sudo access on all nodes
• Basic knowledge of Linux commands

2.2 Node Configuration

• Master Node: k3s-master
• Worker Node 1: k3s-worker-01
• Worker Node 2: k3s-worker-02

---

3. Master Node Installation

3.1 Install K3s on Master Node

Connect to your master server and run:

curl -sfL https://get.k3s.io | sh -

3.2 Verify Installation

Check that K3s is running:

sudo systemctl status k3s

Check the node status:

sudo kubectl get nodes

You should see the master node in "Ready" status.

3.3 Configure kubectl Access (Without sudo)

Option A: Using KUBECONFIG Environment Variable

Step 1: Search for existing KUBECONFIG configurations

grep -r "KUBECONFIG" ~/

If you find a line like export KUBECONFIG=/etc/rancher/k3s/k3s.yaml in files like ~/.bashrc, you need to remove or comment it out.

Step 2: Edit the configuration file

nano ~/.bashrc

Comment out or remove the line:

# export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Step 3: Apply changes

source ~/.bashrc

Step 4: Configure kubectl properly

Create the kube directory and copy the config:

mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $(whoami):$(whoami) ~/.kube/config
chmod 600 ~/.kube/config

Step 5: Set KUBECONFIG permanently

echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
source ~/.bashrc

Step 6: Test kubectl access

kubectl get nodes

You should now see your nodes without using sudo.

Option B: Using a Custom Group (Recommended Alternative)

This method is cleaner and avoids permission issues.

Step 1: Create a new admin group

# Create the k8s-admins group
sudo groupadd k8s-admins

# Add your current user to the group
sudo usermod -aG k8s-admins $(whoami)

Step 2: Log out and log back in

⚠️ IMPORTANT: You must log out and log back in for group changes to take effect.

Verify group membership:

groups

You should see k8s-admins in the list.

Step 3: Reinstall K3s with proper permissions

curl -sfL https://get.k3s.io | \
  INSTALL_K3S_EXEC="--write-kubeconfig-mode 644 \
  --write-kubeconfig-group k8s-admins" sh -

Explanation:

• --write-kubeconfig-mode 644: Sets read/write for owner and read for group
• --write-kubeconfig-group k8s-admins: Sets the group owner to k8s-admins

Step 4: Configure kubectl environment

# Create .kube directory
mkdir -p ~/.kube

# Copy configuration (no sudo needed this time!)
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

# Ensure you own the file
sudo chown $(whoami):$(whoami) ~/.kube/config

Step 5: Test kubectl access

kubectl get nodes

3.4 Get Node Token

Save this token for joining worker nodes:

sudo cat /var/lib/rancher/k3s/server/node-token

Example output:

K10abcdef1234567890::server:abcdef1234567890abcdef12

💾 Save this token securely - you'll need it for worker nodes.

---

4. Worker Nodes Installation

4.1 Install K3s Agent on Worker Node 1

Connect to worker-01 and run:

curl -sfL https://get.k3s.io | \
  K3S_URL=https://MASTER_IP:6443 \
  K3S_TOKEN=YOUR_TOKEN_HERE sh -

Replace:

• MASTER_IP: Your master node's IP address
• YOUR_TOKEN_HERE: The token from step 3.4

4.2 Install K3s Agent on Worker Node 2

Connect to worker-02 and run the same command:

curl -sfL https://get.k3s.io | \
  K3S_URL=https://MASTER_IP:6443 \
  K3S_TOKEN=YOUR_TOKEN_HERE sh -

4.3 Verify Cluster

From the master node, check all nodes:

kubectl get nodes

Expected output:

NAME              STATUS   ROLES                  AGE   VERSION
k3s-master        Ready    control-plane,master   10m   v1.28.x+k3s1
k3s-worker-01     Ready    <none>                 5m    v1.28.x+k3s1
k3s-worker-02     Ready    <none>                 3m    v1.28.x+k3s1

---

5. Troubleshooting

5.1 Worker Node Not Appearing in Cluster

If kubectl get nodes only shows the master node, the worker might be pointing to localhost instead of the master IP.

5.1.1 Verify Logs

sudo journalctl -u k3s-agent.service --no-pager

Common error message:

Error starting load balancer: listen tcp 127.0.0.1:6444:
address already in use

5.1.2 Solution 1: Kill Process Blocking Port 6444

Step 1: Find the process using port 6444

sudo lsof -i :6444

Or if lsof is not installed:

sudo netstat -tulpn | grep :6444

Step 2: Kill the process

sudo kill -9 <PID>

5.1.3 Solution 2: Complete Reinstallation (If Step 1 Fails)

Step 1: Stop all K3s services

sudo systemctl stop k3s k3s-agent

Step 2: Uninstall existing K3s

sudo /usr/local/bin/k3s-uninstall.sh
sudo /usr/local/bin/k3s-agent-uninstall.sh

Step 3: Remove all residual files

sudo rm -rf /var/lib/rancher/k3s/ \
  /etc/rancher/k3s/ \
  /var/lib/rancher/ \
  /etc/systemd/system/k3s*.service

Step 4: Reload systemd

sudo systemctl daemon-reload

Step 5: Verify port 6444 is free

sudo ss -tulpn | grep :6444

Should return nothing.

Step 6: Reinstall with correct master IP

curl -sfL https://get.k3s.io | \
  K3S_URL=https://MASTER_IP:6443 \
  K3S_TOKEN=TOKEN_FROM_MASTER sh -

Step 7: Verify service status

sudo systemctl status k3s-agent.service

Step 8: Check from master node

kubectl get nodes

5.1.4 Additional Reference

• Unable to connect worker node to master using K3S

5.2 Cluster Not Responding - Boot Loop Issue

If the cluster is not responding and continuously restarting, check the logs to identify the problem.

5.2.1 Check for Boot Loop Error

View the K3s service logs:

sudo journalctl -u k3s -f

If you see an error like:

level=error msg="controller-manager exited: unknown flag:
--pod-eviction-timeout"

This means the kube-controller-manager, an essential Kubernetes component integrated by K3s, is receiving an argument (--pod-eviction-timeout) that it doesn't recognize.

5.2.2 Root Cause

This error typically occurs due to:

• Version mismatch: In recent versions of Kubernetes (and K3s), the --pod-eviction-timeout flag was removed or replaced
• Legacy configuration: A previous installation or manual configuration file is passing this deprecated flag

5.2.3 Solution: Remove Deprecated Configuration

Step 1: Check if the K3s config file exists

ls /etc/rancher/k3s/config.yaml

Step 2: If the file exists, edit it

sudo nano /etc/rancher/k3s/config.yaml

Step 3: Look for and remove any line containing --pod-eviction-timeout

The flag might be located under:

• controller-manager-args: section
• Or directly as an argument

Step 4: Delete the problematic line and save the file

Step 5: Restart K3s service

sudo systemctl restart k3s

Step 6: Verify the service is running properly

sudo systemctl status k3s
kubectl get nodes

5.2.4 If config.yaml Doesn't Exist

If the config.yaml file doesn't exist or doesn't contain the flag:

1. The flag might be embedded in the systemd service (uncommon with clean K3s installation)
2. Consider a complete reinstallation following the steps in Section 5.1.3

📝 Note: If you find other unfamiliar configurations in config.yaml, they might be remnants from a previous failed installation. Review them carefully or remove the entire file and reinstall K3s.

---

6. Firewall Configuration

6.1 Required Ports

Configure UFW firewall on all nodes with the necessary ports.

Reference Guide:

• K3s Firewall Security Setup - Step 3
• How to Open and Close Ports in Linux Ubuntu (Video)

6.2 Master Node Firewall

sudo ufw allow 6443/tcp      # Kubernetes API
sudo ufw allow 10250/tcp     # Kubelet metrics
sudo ufw allow 8472/udp      # Flannel VXLAN
sudo ufw allow 2379:2380/tcp # etcd
sudo ufw allow 22/tcp        # SSH
sudo ufw enable

6.3 Worker Nodes Firewall

sudo ufw allow 10250/tcp  # Kubelet metrics
sudo ufw allow 8472/udp   # Flannel VXLAN
sudo ufw allow 22/tcp     # SSH
sudo ufw enable

6.4 Verify Firewall Status

sudo ufw status verbose

⚠️ Important: If you accidentally disable port 22 (SSH), you'll need physical access to the server or web console (iDRAC, iLO, etc.) to re-enable it.

6.5 Test Network Connectivity

Test connectivity between nodes using ping:

# From master to worker-01
ping WORKER_01_IP

# From master to worker-02
ping WORKER_02_IP

# From worker-01 to master
ping MASTER_IP

All nodes should respond successfully.

---

7. SSH Configuration

7.1 Configure SSH Keys for Secure Connection

Set up SSH key authentication between all three nodes for secure, password-less access.

Steps:

1. Generate SSH key on the master node (if not already created):

ssh-keygen -t rsa -b 4096

2. Copy SSH key to worker nodes:

ssh-copy-id user@WORKER_01_IP
ssh-copy-id user@WORKER_02_IP

3. Test SSH connection:

ssh user@WORKER_01_IP
ssh user@WORKER_02_IP

You should now be able to connect without entering a password.

7.2 Verify SSH Connections

Confirm that all nodes can communicate securely via SSH.

---

8. File Transfer Between Nodes

8.1 Using SCP Command

Transfer files from your local machine to servers or between servers using scp.

Syntax:

scp -r /local/path/folder user@DESTINATION_IP:/remote/path/

Example: Transfer test scripts from local machine to master node

scp -r /local/path/test-scripts \
  admkube01@192.168.1.10:/home/admkube01/

8.2 Working with VSCode

If you're developing scripts locally in VSCode:

1. Create your test scripts locally
2. Use scp command to transfer to the server
3. Execute on the server

Reference Guide:

• Copy files between servers with SCP command

---

9. Container Management

9.1 Workflow for Container Operations

Use the appropriate tool depending on your needs:

1. For cluster operations:

kubectl [command]  # Run from master node

2. For container-level debugging:

sudo k3s ctr [command]    # K3s container runtime
sudo k3s crictl [command] # CRI debugging tool

Run these commands on each node as needed.

9.2 Common kubectl Commands

# Get all nodes
kubectl get nodes

# Get all pods
kubectl get pods -A

# Get deployments
kubectl get deployments

# Describe a pod
kubectl describe pod <pod-name>

# View logs
kubectl logs <pod-name>

9.3 Common crictl Commands (Debugging)

# List all containers
sudo k3s crictl ps

# List all images
sudo k3s crictl images

# Inspect a container
sudo k3s crictl inspect <container-id>

---

10. References

10.1 Official Documentation

• K3s Official Documentation
• Kubernetes Documentation

10.2 Installation Guides

• How to set up a K3s Kubernetes Cluster on Ubuntu 22.04

10.3 Troubleshooting

• Unable to connect worker node to master using K3S

10.4 Utilities

• Copy files between servers with SCP
• How to Open and Close Ports in Linux Ubuntu

---

11. Appendix: Quick Command Reference

11.1 Master Node Setup

# Install K3s
curl -sfL https://get.k3s.io | sh -

# Get node token
sudo cat /var/lib/rancher/k3s/server/node-token

# Configure kubectl
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $(whoami):$(whoami) ~/.kube/config

11.2 Worker Node Setup

# Install K3s agent
curl -sfL https://get.k3s.io | \
  K3S_URL=https://MASTER_IP:6443 \
  K3S_TOKEN=YOUR_TOKEN sh -

# Check agent status
sudo systemctl status k3s-agent

11.3 Cluster Verification

# Check nodes (from master)
kubectl get nodes

# Check pods
kubectl get pods -A

# Check services
kubectl get services -A

11.4 Troubleshooting

# View agent logs
sudo journalctl -u k3s-agent.service --no-pager

# Check port usage
sudo ss -tulpn | grep :6444

# Restart services
sudo systemctl restart k3s-agent

---

Document Version: 1.0
Last Updated: October 2025
Status: Production Ready