docker_ollama.md

1. 拉取镜像

docker pull ollama/ollama

2. 创建volume

docker volume create ollama

3.配置nvidia-container-toolkit

(1)添加 NVIDIA 仓库
distribution=rhel8.6  # OpenEuler 兼容 RHEL
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

(2)安装 nvidia-container-toolkit
sudo yum clean all
sudo yum install -y nvidia-container-toolkit

(3)配置 Docker 运行时
sudo nvidia-ctk runtime configure --runtime=docker

(4)重启 Docker
sudo systemctl restart docker

4. 创建容器

docker run -d   --name ollama   --privileged   --device=/dev/nvidia0:/dev/nvidia0   --device=/dev/nvidiactl:/dev/nvidiactl   --device=/dev/nvidia-uvm:/dev/nvidia-uvm   -p 11434:11434 --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=0 -v ollama:/root/.ollama   -v /data/models:/data/models   --restart always   ollama/ollama

5. 下载模型

docker exec -it ollama ollama pull qwen3:8b
# 补充一下，若要下载Qwen3-Embedding-8B:F16
docker exec -it ollama ollama pull dengcao/Qwen3-Embedding-8B:F16
# 补充，若想需要不思考的模型
docker exec -it ollama ollama pull hoangquan456/qwen3-nothink:8b

6. 进行测试

docker exec -it ollama ollama run qwen3:8b "你好"
docker exec -it ollama ollama run hoangquan456/qwen3-nothink:8b "你好"

若要测试Qwen3-Embedding-8B

curl http://localhost:11434/api/embeddings -d '{ "model": "dengcao/Qwen3-Embedding-8B:F16", "prompt": "这是一个测试文本" }'