Skip to content

Instantly share code, notes, and snippets.

@steffenba

steffenba/longhorn-replicas-eviction.md

Created September 23, 2024 15:14
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save steffenba/1bf79f56e1b36859fc6de637c62e48dc to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save steffenba/1bf79f56e1b36859fc6de637c62e48dc to your computer and use it in GitHub Desktop.
Download ZIP
Longhorn: Cannot evict pod as it would violate the pod's disruption budget
Raw
longhorn-replicas-eviction.md

When upgrading (k3s) Kubernetes Nodes using Ansible to drain the nodes, the following error appears:

error when evicting pods/"instance-manager-xxxx" -n "longhorn" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.

I am using the CNPG-Operator to provision a few Postgres-Clusters in Kubernetes for my applications. These use Longhorn as a single-replica storage backend. More than one replicas would be pointless, since postgres itself manages data replication.

Since longhorn will not allow single-replica PVs to be evicted when a node is drained by default, we need to change that behaviour for node upgrades.

The general explanation can be found here: https://longhorn.io/docs/1.6.0/references/settings/#node-drain-policy

When using the helm chart, change the following parameter (Default value : block-if-contains-last-replica )

defaultSettings.nodeDrainPolicy: block-for-eviction-if-contains-last-replica

Now draining nodes should work fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment