QNAP NFSv4+ through opnsense not working

qnap_nfs4_opnsense.md

Preface

I wanted to use my QNAP NAS as an NFSv4 Kubernetes nfs-csi target.

The Cluster is running on RHEL 9. Mounting from either the cluster hosts, or other RHEL 9 hosts didn't work.

The error was:

mount.nfs: mounting nas.example.com:/kubernetescsinfsslow failed, reason given by server: No such file or directory

Between the Cluster and my NAS, there's an OPNSense appliance.

Solution

Apart from opening ports 111 und 2049 TCP/UDP on the OPNSense firewall from all k8s workers to my NAS, I also did the following:

In my storageclass I added:

mountOptions:
  - nfsvers=4.1

Full example:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi
provisioner: nfs.csi.k8s.io
parameters:
  server: nas.example.com
  share: /kubernetescsinfsslow
  mountPermissions: "755"
  onDelete: retain
  subDir: "${pvc.metadata.namespace}/${pvc.metadata.name}"
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - nfsvers=4.1

I also ensure that on the QNAP NAS the following options were set:

Control Panel > Network & File Services -> Win/MAC... -> NFS Service -> Enable NFS, NFSv4, NFSv4.1

And the important part I was missing: Control Panel -> Shared Folders -> kubernetescsinfsslow (Edit Shared Folder Permission) -> NFS host access -> I added all 3 Kubernetes Workers AND the OPNSense Host IP itself.

For some reason that I don't understand, OPNSense itself also appears in the Firewall-Logs as requesting connections to the NAS on 2049. By adding it to the allowed hosts on QNAP everything started working.