MortezaJavadian/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Docker Course

Welcome To The Course!
Docker Version & System Information

Use these commands to verify your Docker setup:
docker version
Shows the client and server (Engine) versions:
docker info
Displays detailed information about your Docker environment (containers, images, storage, etc.).
Running Docker Without Root

By default only root (or users in the docker group) can access the Docker socket. To let a non-root user run Docker:
# Add your user to the docker group
sudo usermod -aG docker <your-user>

# Apply the new group without logging out
sudo newgrp docker

# (Optional) Verify the docker group in /etc/group
cat /etc/group | grep docker

Note: You may need to log out and back in (or reboot) for the group change to take full effect.

Pulling Images & Listing Local Images

Pulling an Image

Use docker pull to download an image from a registry (default: Docker Hub):
docker pull centos:7
When you use docker run with an image name, Docker will:

Check if the image exists locally.
If not found, it automatically pulls it from the registry.

Listing Local Images

docker images
Displays all locally available images:

REPOSITORY – image name
TAG – version/tag
IMAGE ID – unique identifier
SIZE – image size

# Optional: list only image IDs
docker images -q
Interactive Containers

Run a CentOS container in interactive mode:
# Start a shell session
docker run -it centos /bin/bash
# — or —
docker run -it centos
# Why do both commands have the same output? becuase main process of centos is /bin/bash

# Run the top command
docker run -it centos top

# Ping Google DNS
docker run -it centos ping 8.8.8.8

-i keeps STDIN open
-t allocates a pseudo-TTY

Inside the container:

Press Ctrl + d to exit and stop the container
Press Ctrl + pq to detach without stopping

Inspecting Docker Processes

You can use ps to find Docker-related processes:
# Show all processes with “docker” in their name
ps aux | grep docker

# Show all processes with “container” in their name
ps aux | grep container
Common Docker processes you’ll see:

dockerd – the Docker daemon
containerd – core container runtime
docker-containerd-runc – low-level OCI runtime shim
docker-containerd-shim – keeps containers running after daemon exits

(Optional) To check the Docker service status:
systemctl status docker
Container Lifecycle & Listing

Every container have a lifecycle with a main process and a container stops when its main process ends(see the COMMAND column in docker ps).
# List running containers
docker ps

Show these columns: CONTAINER ID, IMAGE, COMMAND, CREATED, STATUS, PORTS, NAMES

The switches of the docker ps
# List exited containers
docker ps -f status=exited

# List all containers (running + stopped)
docker ps -a

# Show the last 2 containers (by creation time)
docker ps -n 2

# Show the most recently created container
docker ps -l

# Display container sizes (first = filesystem, second = virtual size)
docker ps -s

# Show only container IDs (Useful)
docker ps -q

# Example: remove all exited containers
docker rm $(docker ps -q -f status=exited)
Removing Containers


Remove a stopped container
docker rm <container>


Force-remove a running or stopped container
docker rm -f <container>


Force-remove all containers
docker rm -f $(docker ps -qa)


Remove a container and its associated volumes
docker rm -v <container>


Enabling Live Restore

Live Restore keeps your containers running and stay up when the Docker daemon stops or restarts.
# See that containers stop when daemon stops
systemctl stop docker
# You can check with `ps aux` that all containers go down

# Prepare daemon config
sudo mkdir -p /etc/docker
sudo echo "
{
  "live-restore": true
}
" >> /etc/docker/daemon.json

# Start (or restart) Docker with live-restore enabled
sudo systemctl start docker
# — or —
sudo systemctl restart docker

# Verify setting
docker info | grep "Live Restore"
# Should show: Live Restore Enabled: true

Now you can test and see that containers running and stay up when the Docker daemon stops or restarts.

Resource Constraints

Limit container resources at runtime:
# Set memory to 400 MiB and memory-swap to 1 GiB (400 MiB RAM + 600 MiB swap)
docker run -itd \
  -m 400M \
  --memory-swap 1G \
  centos

# Pin container to use just CPU 0
docker run -itd \
  --cpuset-cpus="0" \
  centos

# Limit container writable layer to 5 GiB
docker run -itd \
  --storage-opt size=5G \
  centos

-m/--memory => defines the maximum RAM.
--memory-swap => sets total RAM+swap (use -1 for unlimited swap).
--cpuset-cpus => binds the container to specific CPU cores.
--storage-opt size => limits the container’s filesystem size, but it only works with the overlay2 storage driver on an xfs filesystem with pquota enabled.

You can see and verify your resource limits with docker inspect:
docker inspect <container> \
  --format='
  Memory Limit   : {{.HostConfig.Memory}} bytes
  Memory + Swap  : {{.HostConfig.MemorySwap}} bytes
  CPU Set        : {{.HostConfig.CpusetCpus}}
  Storage Size   : {{index .HostConfig.StorageOpt "size"}}'
Monitoring Container Metrics and Resource

Use docker stats to view real-time resource usage:
# Live update for all running containers
docker stats

# For a specific container
docker stats <container_name_or_id>

# Single snapshot
docker stats --no-stream
Run Heavy CPU Load Test

Start a CentOS container with CPU and memory limits:
docker run -it --cpuset-cpus="0" -m 300M --name test-centos centos

Inside the container:

Check CPU and memory and swap info:
cat /sys/fs/cgroup/cpuset.cpus
cat /sys/fs/cgroup/memory.max
cat /sys/fs/cgroup/memory.swap.max

Use this command to generate heavy CPU load in the background:
yes > /dev/null &

Run top to see processes.


Exit the container without stop container (Ctrl + pq).
On the host, run docker stats to see how “test-centos” used CPU and memory.

Tagging Docker Images

Assign or change an image’s name and tag:
# Rename or retag an existing image
docker tag <old_image>:<old_tag> <new_image>:<new_tag>
Handle dangling images (<none>:<none>) by using the image ID:
# Tag a dangling image
docker tag <image_id> <new_image>:<new_tag>
List dangling images before tagging:
docker images -f “dangling=true”
Searching for Images

Use docker search to find images on Docker Hub:
docker search <image_name>
Inspecting Docker Objects

docker inspect returns JSON details for containers, images, networks, volumes, and more.
# Inspect a container
docker inspect <container_name_or_id>

# Inspect an image
docker inspect <image_name_or_id>
Key fields you might check:

State.Status – running, exited, etc.
NetworkSettings.IPAddress – container IP.
HostConfig.Binds – volume mounts.
...

# Example: show only the IP address of a container
docker inspect -f '{{.NetworkSettings.IPAddress}}' my-container
Attaching to a Running Container

Use docker attach to connect your terminal’s stdin, stdout, and stderr to a running container:
docker attach <container_name_or_id>

You’ll see the container’s live console output.
Keyboard input is sent directly to the container’s main process.
To detach without stopping the container, press Ctrl + pq.

Running Commands in a Container

docker exec starts a new process inside a running container.

If this process exits, the container keeps running (because the main process is still alive).
# Run a command (example: /bin/bash shell)
docker exec -it mycontainer /bin/bash

# Run a simple command
docker exec -it mycontainer echo "hello"

# Set working directory for the command
docker exec -it -w /var mycontainer ls
# - Lists contents of /var inside the container

# Set an environment variable for the command
docker exec -it -e myvar=123 mycontainer bash
# - Inside bash, 'echo $myvar' will output 123
Note: Use docker exec instead of docker attach when you just want to run extra commands without interfering with the main process.
Copying Files Between Host and Container

docker cp lets you copy files or directories to or from a container.
# Copy a file from host to container
docker cp myfile mycontainer:/path/in/container

# Copy a file from container to host
docker cp mycontainer:/path/in/container/myfile .

Works for both running and stopped containers.
Paths can be absolute or relative.

Starting, Stopping, Restarting, and Killing Containers

Stop a Container Gracefully

docker stop -t 20 mycontainer

Sends a SIGTERM signal, giving the container up to 20 seconds to exit before forcing SIGKILL.

Wait for a Container to Stop

docker wait mycontainer

Blocks until the container stops and return its exit code.

Start a Stopped Container

# Attach to stdout/stderr (no stdin)
docker start -a mycontainer

# Attach with stdin, stdout, stderr
docker start -ai mycontainer
Restart a Container

docker restart -t 20 mycontainer

Stops (gracefully) and then starts the container again.

Kill a Container Immediately

docker kill mycontainer

Sends a SIGKILL signal (no cleanup time).

stop vs kill:

stop → Graceful shutdown, gives time to finish work.
kill → Immediate termination. (Forceful)

Note:

Restarting or starting a container does not remove its files.
Files are deleted only when the container is removed (docker rm).

Setting Environment Variables at Runtime

Use -e (or --env) to set environment variables when starting a container:
docker run -e key=value <image_name>
You can set multiple variables:
docker run -e VAR1=foo -e VAR2=bar centos env
Viewing Container Logs

Use docker logs to read a container’s stdout and stderr output.
# Basic logs
docker logs mycontainer

# Follow logs in real-time
docker logs -f mycontainer

# Show timestamps
docker logs -t mycontainer

# Follow logs with timestamps
docker logs -ft mycontainer

# Show logs since a certain time
docker logs --since 2025-08-14T15:00:00 mycontainer
docker logs --since 10m mycontainer   # last 10 minutes

# Show logs until a certain time
docker logs --until 2025-08-14T16:00:00 mycontainer
docker logs --until 5m mycontainer    # until 5 minutes ago
Monitoring Docker Events

docker events streams real-time events from the Docker daemon (such as container start/stop, image pull, network changes).
# Show all events as they happen
docker events

# Show events since a specific time
docker events --since '2025-08-14T15:00:00'
docker events --since 10m   # last 10 minutes

# Filter by event type or object
docker events --filter 'event=stop'              # only stop events
docker events --filter 'container=mycontainer'   # events for a specific container
docker events --filter 'type=network'            # only network-related events
Pausing and Unpausing Containers

Pause a container: Suspends all processes in the container using cgroups freezer.
docker pause mycontainer

The container remains running but its processes are frozen (no CPU execution).

Unpause a container: Resumes the processes.
docker unpause mycontainer
Note: Useful for temporarily halting resource usage without stopping the container.
Automatically Remove Container on Exit

Use --rm with docker run to automatically delete the container when it stops.
docker run --rm <image_name>

Saves cleanup work for short-lived or test containers.

Creating an Image from a Container with committing

docker commit saves the current state of a container as a new image.
Change in container and commit

# Copy a file into the container first
docker cp file1 mycontainer:/path/in/container

# Commit the container to a new image
docker commit mycontainer newimagename:X

# Run a container from the new image
docker run newimagename:X
Change the default command on commit

docker commit --change='CMD ["/bin/sh"]' mycontainer newimagename:X
Note: docker commit is useful for quick snapshots, but for reproducible builds and version control, it’s better to use a Dockerfile and docker build.
Renaming a Container

Use docker rename to change a container’s name without recreating it.
docker rename <old_name> <new_name>
Note: The container ID stays the same — only the name changes.
Saving and Loading Images

Save an Image to a File

docker save -o myimage.tar.gz myimage:tag

Exports the image as a .tar.gz (or .tar) file for backup or transfer.

Load an Image from a File

docker load < myimage.tar.gz

Imports the image back into Docker (used only for images saved with docker save).

Note: docker save/load works for images.
Exporting and Importing Containers

Export a Container to a File

# Using output redirection
docker export mycontainer > myimage.tar

# Using --output option
docker export --output="myimage.tar" mycontainer

Saves the container’s filesystem (without history, layers, or metadata).

Import a Container File as an Image

docker import myimage.tar

Creates a new dangling image (<none>:<none>).
Tag it with a proper name:

docker tag <image_id> newname:newtag
Running the Imported Image

Since docker export/import delete metadata (including the default CMD), you must set the command manually:
docker run -it newname:newtag /bin/bash
Note: Use docker save/load for full image backups with history. Use docker export/import for a clean snapshot of a container’s filesystem.
Docker Logging Drivers

Docker uses logging drivers to handle container logs.

Changing the logging driver does not change the output of docker logs — it only changes how logs are stored or where they are sent.
Run with a Specific Logging Driver

docker run --log-driver <driver_name> centos
Common Logging Drivers


none – disables logging entirely.
local – stores logs in a binary format on the host (efficient).
json-file – default logging driver; stores logs as JSON text files.
syslog – sends logs to the system’s syslog daemon.
journald – sends logs to systemd-journald.
gelf – sends logs to Graylog Extended Log Format endpoints.
fluentd – sends logs to a Fluentd collector.
awslogs – sends logs to Amazon CloudWatch Logs.
splunk – sends logs to Splunk.
etwlogs – sends logs to Event Tracing for Windows.
gcplogs – sends logs to Google Cloud Logging.

Note: Default log storage location for local and json-file:
/var/lib/docker/containers/<container-id>/<container-id>-json.log

Check the logging driver of a container

docker inspect -f '{{.HostConfig.LogConfig.Type}}' mycontainer
Blocking vs Non-Blocking Logging

# Non-blocking mode with max buffer size
docker run --log-driver json-file \
  --log-opt mode=non-blocking \
  --log-opt max-buffer-size=4m \
  centos

Blocking – container waits until logs are written.
Non-blocking – logs are buffered; if full, old logs are dropped.

Custom Log Tags

You can add custom metadata to logs that show when use docker logs:
docker run --log-driver json-file \
  --log-opt tag="{{.ImageName}}/{{.Name}}" \
  centos

Available placeholders include .ID, .Name, .ImageName, .ImageID, .DaemonName, etc.

Types of Docker Storage

Docker supports three main storage types for persisting or sharing data between host and containers.
1. Volumes


Managed by Docker.
Stored under:

/var/lib/docker/volumes/


Best choice for persistent data (backups, migration).

2. Bind Mounts


Maps a host file or directory into the container.
Path on the host is fully controlled by you.
Changes affect both host and container immediately.

3. tmpfs Mounts

Tmpfs mounts store data temporarily in host memory (never written to disk).

Data is lost when the container stops.
Useful for سensitive and security data like passwords or temporary files.
Cannot be shared between containers.
Supported only on Linux hosts.

Volume Mounts

Create a volume with a label

docker volume create myvol --label testvol
List volumes

# Shows all volumes on the host
docker volume ls
Run container with volumes

# Mount the volume to a path inside the container 
docker run -d -v myvol:/mydata myimage

# Mount the same volume to two paths inside the container
docker run -d -v myvol:/mydata1 -v myvol:/mydata2 myimage

# Alternative syntax with --mount (recommended)
docker run -itd --mount type=volume,source=myvol,target=/mydata myimage
Filter and inspect volumes

# List volumes by name
docker volume ls -f name=myvol

# Show detailed info about a volume
docker volume inspect myvol
List of Volumes & Containers Mapping

# Lists all containers that are using the volume named 'myvol'
docker ps -a --filter volume=myvol

# Shows detailed mount points (volumes, bind mounts) of the specified container
docker inspect --format="{{.Mounts}}" mycontainer
Create a volume with NFS

docker volume create \
  --driver local \
  --opt type=nfs \
  --opt o=addr=192.168.100.10,rw \
  --opt device=:/data/ \
  mynfsvol

Creates a Docker volume that mounts an NFS export from 192.168.100.10:/data/.
The NFS server must be reachable and allow the host to mount the export.
If the server is not available, Docker still creates the volume without error, but attaching it to a container will fail until the NFS server is accessible.

Remove volumes

# Remove a specific volume
docker volume rm myvol

# Remove all unused volumes
docker volume prune
Note:

Deleting a volume removes all its data.
Containers using that volume must be stopped or removed first. If you don’t stop or remove the containers using the volume first, Docker will not allow the volume to be removed and will return an error.

Bind Mounts and Permissions

Bind mounts map a host directory into a container. Permissions control what the container can do.
Read-only root filesystem with writable bind mount

docker run -itd --read-only \
  --mount type=bind,source=/mnt,target=/mnt \
  myimage

Container’s root filesystem is read-only.
/mnt in host is writable.

Using :ro or :rw with bind mount and -v

# Read-only mount
docker run -v /host/path:/container/path:ro myimage

# Read-write mount (default)
docker run -v /host/path:/container/path:rw myimage
Recommended syntax with --mount

docker run -itd \
  --read-only \
  --mount type=bind,source=/mnt,target=/mnt,rw \
  myimage

Container’s root filesystem is read-only.
/mnt in host and container is writable.

Note: Use --mount for complex setups; easier to read and avoids mistakes compared to -v.
Tmpfs Mounts

Using --tmpfs

docker run -itd --tmpfs /tmp myimage
Using --mount

docker run -itd --mount type=tmpfs,destination=/tmp myimage
Note: Tmpfs is fast and secure for temporary in-memory storage.
Docker Networks

Docker networks let containers communicate with each other and with the host.

They provide isolation, security, and flexible connection options.
List Networks

# This shows all existing networks (bridge, host, none, or custom).
docker network ls
Note: By default, Docker creates this three networks: bridge, host, none
Types of Network Drivers


bridge (default):
Creates a private network on the host. Containers on the same bridge can talk to each other.
docker run -it --network bridge myimage


host:
Shares the host’s network stack. The container uses the same IP as the host. No isolation.
docker run -it --network host myimage


none:
No network is assigned. Useful for security or manual network setups.
docker run -it --network none myimage


overlay:
Used for multi-host Docker Swarm. Connects containers across different machines (hosts).
# Create a custom overlay network
docker network create -d overlay my_overlay

# Run a container attached to the custom overlay network
docker run -it --network my_overlay myimage


macvlan:
Gives each container its own MAC and IP, making it appear as a physical device on the network.
# Create a custom macvlan network with subnet and gateway
docker network create -d macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  my_macvlan

# Run a container attached to the custom macvlan network
docker run -it --network my_macvlan myimage


Inspecting Networks


Inspect a container’s network:

Shows the container’s network configuration like Gateway, IP, network type, and MAC address (inside "NetworkSettings" section).
docker inspect mycontainer


Inspect a network:
Displays details of the network and the containers connected to it (inside "Containers" section).
docker network inspect my_network


Checking Networks in Host and Container


On Docker host:

You can see Docker networks (like docker0) and container interfaces using:
ip a


Inside a container:


If the ip command exists:
ip a


If not available, use:
cat /proc/net/dev


This shows network interfaces inside the container so you can verify the container’s network setup.
No results found